System and method for performing flash caching of dynamically generated objects in a data communication network

ABSTRACT

The present invention is directed towards a method and system for providing a technique referred to as flash caching to respond to requests for an object, such as a dynamically generated object, from multiple clients. This technique of the present invention uses a dynamically generated object stored in a buffer for transmission to a client, for example in response to a request from the client, to also respond to additional requests for the dynamically generated object from other clients while the object is stored in the buffer. Using this technique, the present invention is able to increase cache hit rates for extremely fast changing dynamically generated objects that may not otherwise be cacheable.

RELATED APPLICATIONS

This present application claims priority to U.S. patent application Ser.No. 11/169,002 entitled “METHOD AND DEVICE FOR PERFORMING INTEGRATEDCACHING IN A DATA COMMUNICATION NETWORK,” filed Jun. 29, 2005, and U.S.patent application Ser. No. 11/039,946, entitled “SYSTEM AND METHOD FORESTABLISHING A VIRTUAL PRIVATE NETWORK,” filed Jan. 24, 2005, both ofwhich are incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates generally to caching data in a network. Inparticular, the present invention relates to a system and method forperforming flash caching of a dynamically generated object to respond torequests for the dynamically generated object from a plurality ofclients.

BACKGROUND

The growth rate of network traffic continues to strain theinfrastructure that carries that traffic. Various solutions have arisento permit network operators to handle this increasing problem, includingthe development of caching technology. With traditional caching, staticcontent can be reused and served to multiple clients without burdeningserver infrastructure. Additionally, cache memories permit staticcontent to be stored closer to the end user, thereby improving responsetime while at the same time reducing server infrastructure burden.Lowered response times and lowered server infrastructure load reducesbandwidth and the processing requirements of such infrastructure.

However, an increasing amount of the content delivered across networksis dynamically generated, including a large percentage of networktraffic created by enterprise computing solutions and complex internetapplications. Dynamically generated content is content generated by theserver at the time an object is requested, and is often based on inputsreceived from the client. Therefore, it frequently changes both throughtime and with respect to inputs made to the generating system. Commonexamples of dynamic content include where a stock quotation request madeby a client or database searches. In each instance, the response objectis generated in real time following receipt of a specific, clientrequest.

The challenges to caching dynamically generated content are manifold.For example, there are no generally-accepted standards or specificationsfor caching dynamically generated content. Since there exists nostandard for designating whether a dynamically generated object may becached, such objects are typically treated as non-cacheable. Anotherchallenge is determining the validity of “freshness” of a dynamicallygenerated object because changes to the underlying data used to generatesuch objects may be irregular and unpredictable.

In addition to the above difficulties, requests for dynamicallygenerated content are also typically more complex than requests forstatic content. Dynamic requests often contain a string of informationthat needs to be processed or parsed by the destination application toidentify applicable parameters for identifying the appropriate objectrelated to such request. These parameters, however, are rarely placed inthe request in a logical or consistent order by the client. To determinewhich of the multitude of dynamically generated objects is identified bythe request, each such request must be normalized (i.e., place theparameters in non-arbitrary order).

Furthermore, matching a request to a dynamically generated objectbecomes a much more complex task with dynamically generated contentbecause certain processing done by the application may need to beduplicated or otherwise anticipated, such as in effect making aneducated guess. This duplication or guessing is necessary to decidewhether an object stored by the cache is appropriate for serving to aparticular incoming request. The complexity arises as a result of thecomplexity of the applications themselves, and also because the contentsof the response can be a function of both the contents of the request,as well as certain other external variables like the user-identity(which may or may not be present in the request), the time of the day,the current state of the user database and a myriad of other factors.

In summary, caching originally developed around the caching of staticobjects. As the Internet and applications becomes more and moredependent upon delivering dynamically generated content, the need hasarisen for a solution that extends the benefits of caching to dynamiccontent, and that solves the variety of challenges such content presentsfor traditional caching technology.

BRIEF SUMMARY OF THE INVENTION

The present invention is directed towards a method and system forproviding a technique referred to as flash caching to respond torequests for an object, such as a dynamically generated object, frommultiple clients. This technique of the present invention uses adynamically generated object stored in a buffer for transmission to aclient, for example in response to a request from the client, to alsorespond to additional requests for the dynamically generated object fromother clients while the object is stored in the buffer. Using thistechnique, the present invention is able to increase cache hit rates forextremely fast changing objects, such as dynamically generated objectsthat would not otherwise be cacheable.

In one aspect, the present invention is related to a method forresponding to a request for a dynamically generated object from aplurality of clients. The method includes receiving, by a device, aresponse from an originating server to a first request of a first clientfor a dynamically generated object. The response includes thedynamically generated object. The method also includes requesting, bythe device, to a packet processing engine to transmit the response tothe first client. The response is held in a buffer while waiting to betransmitted. The method further includes receiving, by the device, froma second client a second request for the dynamically generated objectprior to completing transmission of the response to the first client,and requesting, by the device, to the packet processing engine totransmit the dynamically generated object of the response to the firstclient held in the buffer to the second client.

In one embodiment, the method of the present invention includestransmitting, by the packet processing engine, the response to the firstclient. In another embodiment, the method includes transmitting, by thepacket processing engine, the dynamically generated object to the secondclient in response to the second request. In some embodiments, themethod includes removing, by the packet processing engine, the responseform the buffer after transmitting the response to one of the firstclient or the second client. In some embodiments, the buffer used by thepresent invention comprises a TCP/IP buffer or a portion of a TCP/IPstack of the device. The device may have a single TCP/IP stack fortransmitting to the first client and the second client, and the bufferis associated with the single TCP/IP stack.

In some embodiments of the present invention, the method includesreceiving, by the device, multiple requests from multiple clients forthe dynamically generated object, and requesting, by the device, totransmit the dynamically generated object of the response to the firstclient held in the buffer to the multiple clients. In one embodiment,the packet processing engine holds the response to the first clientbased on one or more characteristics related to a connection of thefirst client to a network In another embodiment, the device queues thesecond request instead of transmitting the second request to theoriginating server. In one embodiment, the device responds to the secondrequest using the response of the first request held in the buffer.

In some embodiments, the method of the present invention removes thedynamically generated object from the buffer after transmitting thedynamically generated object to the first client and the second client.In one embodiment, the dynamically generated object is identified asnon-cacheable, such as when served from an originating server. In otherembodiments of the method, the device determines the second request isfor the dynamically generated object of the first request via an objectdeterminant associated with the second request.

In some embodiments, the device may be an appliance, a network device ora computing device in communication between the first client and theoriginating server. In further embodiments, the device comprise a cachein communication with the packet processing engine. In one embodiment,the method includes requesting, by the cache, the packet processingengine to transmit one of the response to the first client or thedynamically generated object of the first client to the second client.

In another aspect, the present invention is related to a system forresponding to a request for a dynamically generated object from aplurality of clients. The system includes a device receiving a responsefrom an originating server to a first request of a first client for adynamically generated object, the response comprising the dynamicallygenerated object, the device comprising, and a buffer for holding one ormore network packets waiting to be transmitted from the device. Thedevice also includes a packet processing engine receiving a request fromthe device to transmit the response to the first client; the bufferholding one or more network packets comprising the response. The devicereceives from a second client a second request for the dynamicallygenerated object prior to completing transmission of the response to thefirst client, and requests the packet processing engine to transmit thedynamically generated object of the response held in the buffer to thesecond client.

In one embodiment of the system of the presenting invention, the packetprocessing engine transmits from the buffer the response to the firstclient. In another embodiment, wherein the packet processing enginetransmits from the buffer the dynamically generated object to the secondclient in response to the second request. In some embodiments of thesystem, the packet processing engine removes the response from thebuffer after transmitting the response to one of the first client or thesecond client. Additionally, in other embodiments, the buffer is aTCP/IP buffer or otherwise is a portion of a TCP/IP stack of the device.In some embodiments, the device has a single TCP/IP stack fortransmitting to the first client and the second client the bufferassociated with the single TCP/IP stack.

In another embodiment of the present invention, the device receives frommultiple clients multiples requests for the dynamically generatedobject, and wherein the device requests the packet processing engine totransmit the dynamically generated object of the response to the firstclient held in the buffer to the multiple clients. In one embodiment,the packet processing engine holds the response to the first clientbased on one or more characteristics related to a connection of thefirst client to a network In another embodiment, the device queues thesecond request instead of transmitting the second request to theoriginating server. In some embodiments, the device queues the secondrequest and responds to the second request with the response held in thebuffer for the first client. In additional embodiments, the packetprocessing engine removes the dynamically generated object from thebuffer after transmitting the dynamically generated object to the firstclient and the second client. In one embodiment, the dynamicallygenerated object is identified as non-cacheable, such as by anoriginating server.

In another embodiment of the present invention, the device determinesthe second request is for the dynamically generated object of the firstrequest via an object determinant associated with the second request. Insome embodiments, the device is an appliance, a network device or acomputing device in communication between the first client and theoriginating server. The device may also have a cache in communicationwith the packet processing engine. Additionally, the cache may requestthe packet processing engine to transmit the response to the firstclient or the dynamically generated object of the first client to thesecond client.

The details of various embodiments of the invention are set forth in theaccompanying drawings and the description below.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated herein and form partof the specification, illustrate the present invention and, togetherwith the description, further serve to explain the principles of theinvention and to enable a person skilled in the relevant art(s) to makeand use the invention.

FIG. 1 is a block diagram illustrating an example network environment inwhich an embodiment of the present invention may be implemented;

FIG. 2 is a block diagram illustrating an example architecture of anappliance that performs integrated caching in accordance with anembodiment of the present invention;

FIG. 3A is a flow diagram of steps taken in an embodiment of a method ofthe present invention for integrating device operations with packetprocessing and the packet processing timer;

FIG. 3B is a flow diagram of steps taken in an embodiment of a method ofthe present invention for practicing invalidation granularity techniquesin view of FIG. 3A;

FIG. 4A is a flow diagram of steps taken in an embodiment of a method ofthe present invention using invalidation commands to invalidate staleobjects;

FIG. 4B is a flow diagram of steps taken in an embodiment of a method ofthe present invention incorporating invalidation of groups of objects;

FIG. 4C is a flow diagram of steps taken in an embodiment of a method ofthe present invention wherein a client request is parsed for objectdeterminants;

FIG. 4D is a flow diagram of steps taken in an embodiment of a method ofthe present invention incorporating invalidation of groups of objectsusing object determinants;

FIG. 5. is a flow diagram of steps taken in an embodiment of a method ofthe present invention for providing a flash cache technique;

FIG. 6. is a flow diagram of steps taken in an embodiment of a method ofthe present invention for providing a flash crowd control technique;

FIGS. 7A and 7B are flow diagrams of steps taken in an embodiment of amethod of the present invention for providing entity tag and cachecontrol for an object; and

FIGS. 8A and 8B are block diagrams of embodiments of a computing devicefor practicing an illustrative embodiment of the present invention.

The features and advantages of the present invention will become moreapparent from the detailed description set forth below when taken inconjunction with the drawings, in which like reference charactersidentify corresponding elements throughout. In the drawings, likereference numbers generally indicate identical, functionally similar,and/or structurally similar elements.

DETAILED DESCRIPTION A. Example Network Environment

FIG. 1 illustrates an example network environment 100 in which anembodiment of the present invention may be practiced. As shown in FIG.1, example network environment 100 includes a plurality of clients 102a-102 n, a plurality of servers 106 a-106 n, and an appliance 104, whichmay also referred to as a cache appliance, device, or cache. The servers106 a-106 n originate and manage databases, such as object or relationaldatabases, that provide requested content to the clients 102 a-102 n.For this reason, the servers 106 a-106 n are sometimes referred toherein as “originating servers” because they typically, though notnecessarily, originate the objects forming the requested content. Eachof the clients 102 a-102 n and servers 106 a-106 n may be any type andform of computing device, such as the computing device 800 described inmore detail later in conjunction with FIGS. 8A and 8B. For example, anyof the client 102 a-102 n may be a mobile computing device, such as atelecommunication device, e.g., cellphone or personal digital assistant,or a laptop or notebook computer in addition to any type of desktopcomputer.

Each of the clients 102 a-102 n are communicatively coupled to appliance104 via a public data communication network 108, while appliance 104 iscommunicatively coupled to servers 106 a-106 n via a private datacommunication network 110. In one embodiment, public data communicationnetwork 108 comprises the Internet and private data communicationnetwork 110 comprises an enterprise network. The public datacommunication network 108 and private data communication network 110 canbe any type and form of network, public, private or otherwise, and insome cases, may be the same network.

Although FIG. 1 shows a network 108 and a network 1110 between theclients 102 a-102 n and the servers 106 a-106 n, the clients 102 a-102 nand the servers 106 a-106 n may be on the same network 108 or 110. Thenetworks 108 and 110 can be the same type of network or different typesof networks. The network 108 and/or the network 110 can be a local-areanetwork (LAN), such as a company Intranet, a metropolitan area network(MAN), or a wide area network (WAN), such as the Internet or the WorldWide Web. The network 108 and/or 110 may be any type and/or form ofnetwork and may include any of the following: a point to point network,a broadcast network, a wide area network, a local area network, atelecommunications network, a data communication network, a computernetwork, an ATM (Asynchronous Transfer Mode) network, a SONET(Synchronous Optical Network) network, a SDH (Synchronous DigitalHierarchy) network, a wireless network and a wireline network. Thetopology of the network 108 and/or 100 may be a bus, star, or ringnetwork topology. The network 108 and/or 110 and network topology may beof any such network or network topology as known to those ordinarilyskilled in the art capable of supporting the operations of the presentinvention described herein.

As shown in FIG. 1, the appliance 104 is shown between the public datacommunication network 108 and the private data communication network 110some In other embodiments, the appliance 104 may be located on thepublic data communication network 108, or on the private datacommunication network 110. In other embodiments, the appliance 104 couldbe an integral part of any individual client 102 a-102 n or anyindividual server 106 a-106 n on the same or different network 108, 110as the client 102 a-102 n. As such, the appliance 104 may be located atany point in the network or network communications path between a client102 a-102 n and a server 106 a-106 n.

In accordance with an embodiment of the present invention, the appliance104 includes cache management logic and also includes or has access to astorage medium which it utilizes to implement a cache memory. Usingthese features, appliance 104 monitors object requests made by clients102 a-102 n to any of the servers 106 a-106 n. Objects returned fromservers 106 a-106 n in response to these object requests are stored inthe cache memory by appliance 104. Subsequent requests for the sameobject from any of clients 102 a-102 n are intercepted by appliance 104,which attempts to deliver the object from the cache rather than passingthe request on to servers 106 a-106 n. This provides the dual benefit ofreducing both the time required to respond to requests from clients 102a-102 n and the load on the infrastructure supporting servers 106 a-106n.

In summary, the network environment 100 depicted in FIG. 1 is presentedby way of example only and is not intended to be limiting. Based on theteachings provided herein, persons skilled in the relevant art(s) willreadily appreciate that the present invention may be implemented in anynetwork environment in which object requests and responses aretransferred between nodes of one or more network(s).

B. Example Appliance or Device Architecture

As will be described in more detail herein, in an embodiment of thepresent invention, the appliance 104 integrates caching functionality atthe kernel level of the operating system with one or more otherprocessing tasks, including but not limited to decryption,decompression, or authentication and/or authorization. Such animplementation is illustrated in the commonly owned and co-pending U.S.patent application Ser. No. 11/169,002 entitled “Method and Device forPerforming Integrated Caching in a Data Communications Network,” filedJun. 29, 2005, which is incorporated by reference herein. Such anexample architecture is described herein in accordance with FIG. 2, butthe present invention is not so limited and other architectures may beused in practicing the operations of the present invention describedherein.

FIG. 2 illustrates an example architecture 200 of an appliance 104. Asnoted above, architecture 200 is provided by way of illustration onlyand is not intended to be limiting. As shown in FIG. 2, examplearchitecture 200 consists of a hardware layer 206 and a software layerdivided into a user space 202 and a kernel space 204.

Hardware layer 206 provides the hardware elements upon which programsand services within kernel space 204 and user space 202 are executed.Hardware layer 206 also provides the structures and elements which allowprograms and services within kernel space 204 and user space 202 tocommunicate data both internally and externally with respect toappliance 104. As shown in FIG. 2, the hardware layer 206 includes aprocessing unit 262 for executing software programs and services, amemory 264 for storing software and data, network ports 266 fortransmitting and receiving data over a network, and an encryptionprocessor 260 for performing functions related to Secure Sockets Layerprocessing of data transmitted and received over the network. In someembodiments, the central processing unit 262 may perform the functionsof the encryption processor 260 in a single processor. Additionally, thehardware layer 206 may comprise multiple processors for each of theprocessing unit 262 and the encryption processor 260. Although thehardware layer 206 of appliance 104 is generally illustrated with anencryption processor 260, processor 260 may be a processor forperforming functions related to any encryption protocol, such as theSecure Socket Layer (SSL) or Transport Layer Security (TLS) protocol. Insome embodiments, the processor 260 may be a general purpose processor(GPP), and in further embodiments, may be have executable instructionsfor performing processing of any security related protocol.

Although the hardware layer 206 of appliance 104 is illustrated withcertain elements in FIG. 2, the hardware portions or components ofappliance 104 may comprise any type and form of elements, hardware orsoftware, of a computing device, such as the computing device 800illustrated and discussed in conjunction with FIGS. 8A and 8B furtherherein. In some embodiments, the appliance 104 may comprise a server,gateway, router, switch, bridge or other type of computing or networkdevice, and have any hardware and/or software elements associatedtherewith.

The operating system of appliance 104 allocates, manages, or otherwisesegregates the available system memory into kernel space 204 and userspace 204. In example software architecture 200, the operating systemmay be any type and/or form of Unix operating system although theinvention is not so limited. As such, the appliance 104 can be runningany operating system such as any of the versions of the Microsoft®Windows operating systems, the different releases of the Unix and Linuxoperating systems, any version of the Mac OS® for Macintosh computers,any embedded operating system, any network operating system, anyreal-time operating system, any open source operating system, anyproprietary operating system, any operating systems for mobile computingdevices or network devices, or any other operating system capable ofrunning on the appliance 104 and performing the operations describedherein.

The kernel space 204 is reserved for running the kernel 230, includingany device drivers, kernel extensions or other kernel related software.As known to those skilled in the art, the kernel 230 is the core of theoperating system, and provides access, control, and management ofresources and hardware-related elements of the application 104. Inaccordance with an embodiment of the present invention, the kernel space204 also includes a number of network services or processes working inconjunction with a cache manager 232. sometimes also referred to as theintegrated cache, the benefits of which are described in detail furtherherein. Additionally, the embodiment of the kernel 230 will depend onthe embodiment of the operating system installed, configured, orotherwise used by the device 104.

In one embodiment, the device 104 comprises one network stack 267, suchas a TCP/IP based stack, for communicating with the client 102 a-102 band/or the server 106 a-106 n. In one embodiment, the network stack 267is used to communicate with a first network, such as network 108, and asecond network 110. In some embodiments, the device 104 terminates afirst transport layer connection, such as a TCP connection of a client102 a-102 n, and establishes a second transport layer connection to aserver 106 a-106 n for use by the client 102 a-102 n, e.g., the secondtransport layer connection is terminated at the appliance 104 and theserver 106 a-106 n. The first and second transport layer connections maybe established via a single network stack 267. In other embodiments, thedevice 104 may comprise multiple network stacks, for example 267 and267′, and the first transport layer connection may be established orterminated at one network stack 267, and the second transport layerconnection on the second network stack 267′. For example, one networkstack may be for receiving and transmitting network packet on a firstnetwork, and another network stack for receiving and transmittingnetwork packets on a second network. In one embodiment, the networkstack 267 comprises a buffer 243 for queuing one or more network packetsfor transmission by the appliance 104.

As shown in FIG. 2, the kernel space 204 includes the cache manager 232,a high-speed layer 2-7 integrated packet engine 240, an encryptionengine 234, a policy engine 236 and multi-protocol compression logic238. Running these components or processes 232, 240, 234, 236 and 238 inkernel space 204 or kernel mode instead of the user space 202 improvesthe performance of each of these components, alone and in combination.Kernel operation means that these components or processes 232, 240, 234,236 and 238 run in the core address space of the operating system of thedevice 104. For example, running the encryption engine 234 in kernelmode improves encryption performance by moving encryption and decryptionoperations to the kernel, thereby reducing the number of transitionsbetween the memory space or a kernel thread in kernel mode and thememory space or a thread in user mode. For example, data obtained inkernel mode may not need to be passed or copied to a process or threadrunning in user mode, such as from a kernel level data structure to auser level data structure. In another aspect, the number of contextswitches between kernel mode and user mode are also reduced.Additionally, synchronization of and communications between any of thecomponents or processes 232, 240, 235, 236 and 238 can be performed moreefficiently in the kernel space 204.

In some embodiments, any portion of the components 232, 240, 234, 236and 238 may run or operate in the kernel space 204, while other portionsof these components 232, 240, 234, 236 and 238 may run or operate inuser space 202. In one embodiment, the present invention uses akernel-level data structure providing access to any portion of one ormore network packets, for example, a network packet comprising a requestfrom a client 102 a-102 n or a response from a server 106 a-106 n. Insome embodiments, the kernel-level data structure may be obtained by thepacket engine 240 via a transport layer driver interface or filter tothe network stack 267. The kernel-level data structure may comprise anyinterface and/or data accessible via the kernel space 204 related to thenetwork stack 267, network traffic or packets received or transmitted bythe network stack 267. In other embodiments, the kernel-level datastructure may be used by any of the components or processes 232, 240,234, 236 and 238 to perform the desired operation of the component orprocess. In one embodiment, a component 232, 240, 234, 236 and 238 isrunning in kernel mode 204 when using the kernel-level data structure,while in another embodiment, the component 232, 240, 234, 236 and 238 isrunning in user mode when using the kernel-level data structure. In someembodiments, the kernel-level data structure may be copied or passed toa second kernel-level data structure, or any desired user-level datastructure.

The cache manager 232 may comprise software, hardware or any combinationof software and hardware to provide cache access, control and managementof any type and form of content, such as objects or dynamicallygenerated objects served by the originating servers 106 a-106 n. Thedata, objects or content processed and stored by the cache manager 232may comprise data in any format, such as a markup language, orcommunicated via any protocol. In some embodiments, the cache manager232 duplicates original data stored elsewhere or data previouslycomputed, generated or transmitted, in which the original data mayrequire longer access time to fetch, compute or otherwise obtainrelative to reading a cache memory element. Once the data is stored inthe cache memory element, future use can be made by accessing the cachedcopy rather than refetching or recomputing the original data, therebyreducing the access time. In some embodiments, the cache memory elementnat comprise a data object in memory 264 of device 104. In otherembodiments, the cache memory element may comprise memory having afaster access time than memory 264. In another embodiment, the cachememory element may comprise any type and form of storage element of thedevice 104, such as a portion of a hard disk. In some embodiments, theprocessing unit 262 may provide cache memory for use by the cachemanager 232 of the present invention. In yet further embodiments, thecache manager 232 may use any portion and combination of memory,storage, or the processing unit for caching data, objects, and othercontent.

Furthermore, the cache manager 232 of the present invention includes anylogic, functions, rules, or operations to perform any embodiments of thetechniques of the present invention described herein. For example, thecache manager 232 includes logic or functionality to invalidate objectsbased on the expiration of an invalidation time period or upon receiptof an invalidation command from a client 102 a-102 n or server 106 a-106n. In some embodiments, the cache manager 232 may operate as a program,service, process or task executing in the kernel space 204, and in otherembodiments, in the user space 202. In one embodiment, a first portionof the cache manager 232 executes in the user space 202 while a secondportion executes in the kernel space 204. In some embodiments, the cachemanager 232 can comprise any type of general purpose processor (GPP), orany other type of integrated circuit, such as a Field Programmable GateArray (FPGA), Programmable Logic Device (PLD), or Application SpecificIntegrated Circuit (ASIC).

The policy engine 236 may include, for example, an intelligentstatistical engine or other programmable application(s). In oneembodiment, the policy engine 236 provides a configuration mechanism toallow a user to identifying, specify, define or configure a cachingpolicy. Policy engine 236, in some embodiments, also has access tomemory to support data structures such as lookup tables or hash tablesto enable user-selected caching policy decisions. In other embodiments,the policy engine 236 may comprise any logic, rules, functions oroperations to determine and provide access, control and management ofobjects, data or content being cached by the appliance 104 in additionto access, control and management of security, network traffic, networkaccess, compression or any other function or operation performed by theappliance 104. Further examples of specific caching policies are furtherdescribed herein.

The encryption engine 234 comprises any logic, business rules, functionsor operations for handling the processing of any security relatedprotocol, such as SSL or TLS, or any function related thereto. Forexample, the encryption engine 234 encrypts and decrypts networkpackets, or any portion thereof, communicated via the appliance 104. Theencryption engine 234 may also setup or establish SSL or TLS connectionson behalf of the client 102 a-102 n, server 106 a-106 n, or appliance104. As such, the encryption engine 234 provides offloading andacceleration of SSL processing. In one embodiment, the encryption engine234 uses a tunneling protocol to provide a virtual private networkbetween a client 102 a-102 n and a server 106 a-106 n. In someembodiments, the encryption engine 234 is in communication with theEncryption processor 260. In other embodiments, the encryption engine234 comprises executable instructions running on the Encryptionprocessor 260.

The multi-protocol compression engine 238 comprises any logic, businessrules, function or operations for compressing one or more protocols of anetwork packet, such as any of the protocols used by the network stack267 of the device 104. In one embodiment, multi-protocol compressionengine 238 compresses bi-directionally between clients 102 a-102 n andservers 106 a-106 n any TCP/IP based protocol, including MessagingApplication Programming Interface (MAPI) (email), File Transfer Protocol(FTP), HyperText Transfer Protocol (HTTP), Common Internet File System(CIFS) protocol (file transfer), Independent Computing Architecture(ICA) protocol, Remote Desktop Protocol (RDP), Wireless ApplicationProtocol (WAP), Mobile IP protocol, and Voice Over IP (VoIP) protocol.In other embodiments, multi-protocol compression engine 238 providescompression of Hypertext Markup Language (HTML) based protocols and insome embodiments, provides compression of any markup languages, such asthe Extensible Markup Language (XML). In one embodiment, themulti-protocol compression engine 238 provides compression of anyhigh-performance protocol, such as any protocol designed for appliance104 to appliance 104 communications. In another embodiment, themulti-protocol compression engine 238 compresses any payload of or anycommunication using a modified transport control protocol, such asTransaction TCP (T/TCP), TCP with selection acknowledgements (TCP-SACK),TCP with large windows (TCP-LW), a congestion prediction protocol suchas the TCP-Vegas protocol, and a TCP spoofing protocol.

As such, the multi-protocol compression engine 238 of the presentinvention accelerates performance for users accessing applications viadesktop clients, e.g., Microsoft Outlook and non-Web thin clients, suchas any client launched by popular enterprise applications like Oracle,SAP and Siebel, and even mobile clients, such as the Pocket PC. In someembodiments, the multi-protocol compression engine 238 by executing inthe kernel mode 204 and integrating with packet processing engine 240accessing the network stack 267 is able to compress any of the protocolscarried by the TCP/IP protocol, such as any application layer protocol.

High speed layer 2-7 integrated packet engine 240, also generallyreferred to as a packet processing engine or packet engine, isresponsible for managing the kernel-level processing of packets receivedand transmitted by appliance 104 via network ports 266. The high speedlayer 2-7 integrated packet engine 240 may comprise a buffer for queuingone or more network packets during processing, such as for receipt of anetwork packet or transmission of a network packer. Additionally, thehigh speed layer 2-7 integrated packet engine 240 is in communicationwith one or more network stacks 267 to send and receive network packetsvia network ports 266. The high speed layer 2-7 integrated packet engine240 works in conjunction with encryption engine 234, cache manager 232,policy engine 236 and multi-protocol compression logic 238. Inparticular, encryption engine 234 is configured to perform SSLprocessing of packets, policy engine 236 is configured to performfunctions related to traffic management such as request-level contentswitching and request-level cache redirection, and multi-protocolcompression logic 238 is configured to perform functions related tocompression and decompression of data.

The high speed layer 2-7 integrated packet engine 240 includes a packetprocessing timer 242. In one embodiment, the packet processing timer 242provides one or more time intervals to trigger the processing ofincoming, i.e., received, or outgoing, i.e., transmitted, networkpackets. In some embodiments, the high speed layer 2-7 integrated packetengine 240 processes network packets responsive to the timer 242. Thepacket processing timer 242 provides any type and form of signal to thepacket engine 240 to notify, trigger, or communicate a time relatedevent, interval or occurrence. In many embodiments, the packetprocessing timer 242 operates in the order of milliseconds, such as forexample 100 ms, 50 ms or 25 ms. For example, in some embodiments, thepacket processing timer 242 provides time intervals or otherwise causesa network packet to be processed by the high speed layer 2-7 integratedpacket engine 240 at a 10 ms time interval, while in other embodiments,at a 5 ms time interval, and still yet in further embodiments, as shortas a 3, 2, or 1 ms time interval. The high speed layer 2-7 integratedpacket engine 240 may be interfaced, integrated or in communication withthe encryption engine 234, cache manager 232, policy engine 236 andmulti-protocol compression engine 238 during operation. As such, any ofthe logic, functions, or operations of the encryption engine 234, cachemanager 232, policy engine 236 and multi-protocol compression logic 238may be performed responsive to the packet processing timer 242 and/orthe packet engine 240. Therefore, any of the logic, functions, oroperations of the encryption engine 234, cache manager 232, policyengine 236 and multi-protocol compression logic 238 may be performed atthe granularity of time intervals provided via the packet processingtimer 242, for example, at a time interval of less than or equal to 10ms. For example, in one embodiment, the cache manager 232 may performinvalidation of any cached objects responsive to the high speed layer2-7 integrated packet engine 240 and/or the packet processing timer 242.In another embodiment, the expiry or invalidation time of a cachedobject can be set to the same order of granularity as the time intervalof the packet processing timer 242, such as at every 10 ms

In contrast to kernel space 204, user space 202 is the memory area orportion of the operating system used by user mode applications orprograms otherwise running in user mode. A user mode application may notaccess kernel space 204 directly and uses service calls in order toaccess kernel services. As shown in FIG. 2, user space 202 of appliance104 includes a graphical user interface (GUI) 210, a command lineinterface (CLI) 212, shell services 214, health monitoring program 216,and daemon services 218. GUI 210 and CLI 212 provide a means by which asystem administrator or other user can interact with and control theoperation of appliance 104, such as via the operating system of theappliance 104 and either is user space 202 or kernel space 204. The GUI210 may be any type and form of graphical user interface and may bepresented via text, graphical or otherwise, by any type of program orapplication, such as a browser. The CLI 212 may be any type and form ofcommand line or text-based interface, such as a command line provided bythe operating system. For example, the CLI 212 may comprise a shell,which is a tool to enable users to interact with the operating system.In some embodiments, the CLI 212 may be provided via a bash, csh, tcsh,or ksh type shell. The shell services 214 comprises the programs,services, tasks, processes or executable instructions to supportinteraction with the appliance 104 or operating system by a user via theGUI 210 and/or CLI 212.

Health monitoring program 216 is used to monitor, check, report andensure that network systems are functioning properly and that users arereceiving requested content over a network. Health monitoring program216 comprises one or more programs, services, tasks, processes orexecutable instructions to provide logic, rules, functions or operationsfor monitoring any activity of the appliance 104. In some embodiments,the health monitoring program 216 intercepts and inspects any networktraffic passed via the appliance 104. In other embodiments, the healthmonitoring program 216 interfaces by any suitable means and/ormechanisms with one or more of the following: the encryption engine 234,cache manager 232, policy engine 236, multi-protocol compression logic238, packet engine 240, daemon services 218, and shell services 214. Assuch, the health monitoring program 216 may call any applicationprogramming interface (API) to determine a state, status, or health ofany portion of the appliance 104. For example, the health monitoringprogram 216 may ping or send a status inquiry on a periodic basis tocheck if a program, process, service or task is active and currentlyrunning. In another example, the health monitoring program 216 may checkany status, error or history logs provided by any program, process,service or task to determine any condition, status or error with anyportion of the appliance 104.

Daemon services 218 are programs that run continuously or in thebackground and handle periodic service requests received by appliance104. In some embodiments, a daemon service may forward the requests toother programs or processes, such as another daemon service 218 asappropriate. As known to those skilled in the art, a daemon service 218may run unattended to perform continuous or periodic system widefunctions, such as network control, or to perform any desired task. Insome embodiments, one or more daemon services 218 run in the user space202, while in other embodiments, one or more daemon services 218 run inthe kernel space.

C. Caching of Dynamically Generated Objects

Dynamic content, such as one or more dynamically generated objects, maybe generated by servers, referred to as application or originatingservers 106 a-106 n and/or back-end databases (not shown) that processobject requests from one or more clients 102 a-102 n, local or remote,as depicted in FIG. 1. As those applications or databases process data,including data related to inputs received from clients, the responseobjects served by these databases and applications may change. Priorobjects generated by those applications or databases in an originatingserver will no longer be fresh and therefore should no longer be storedby a cache. For example, given the same set of inputs a dynamicallygenerated object of a first instance may be different than a dynamicallygenerated object of a second instance. In another example, the sameobject may be dynamically generated with a different set of inputs suchthat a first instance of the object is generated differently from asecond instance of the object.

In order to achieve improved network performance, the appliance 104 isdesigned and configured to addresses the problems that arise in cachingdynamically generated content through a variety of methods, as describedin detail below. In some embodiments of the present invention describedherein, the appliance 104 incorporates a set of one or more techniquesfor making the invalidation of dynamically generated content stored inthe cache more efficient and effective. Furthermore, the appliance mayincorporate techniques for performing control and caching for flashcrowds. Cache memories typically store every response to a request foran object as long as such response is not marked as non-cacheable. Asdescribed herein, efficient caching of dynamically generated contentsrequires techniques that enable the timely invalidation of objects inthe cache memory that have undergone a change at the originating server.Timely invalidation allows the cache to avoid serving stale content—atask of particular concern with dynamically generated content,especially where changes to the content occur irregularly. Set forthbelow are a number of techniques to ensure timely invalidation ofdynamically generated content.

1. Integrated Functionality

In one aspect, the present invention is related to techniques ofintegrating functions, logic, or operations of the cache manager 232,policy engine 236, encryption engine 234, and/or the multi-protocolcompression engine 238 with packet processing operations of thehigh-speed layer 2-7 integrated packet engine 240 responsive to thepacket processing timer 242. For example, the operations of the cachemanager 232 can be performed within the time intervals of the packetprocessing timer 242 used for packet processing operations, such as on areceipt or transmit of a network packet. In one embodiment, byintegrating with the packet processing operations and/or using thepacket processing timer, the cache manager 232 of the present inventioncan cache objects with expiry times down to very small intervals oftime, as will be described in further detail below. In otherembodiments, the cache manager 232 responsive to the packet processingtimer 242 can also receive an invalidation command to invalidate anobject within a very short time period of caching the object.

The method 300 depicted in FIG. 3A illustrates one embodiment of atechnique of the present invention for requesting the cache manager 232,policy engine 236, encryption engine 234, and/or the multi-protocolcompression engine 238 to perform an operation during processing or inassociation with the time intervals for processing a network packet bythe high-speed layer 2-7 integrated packet engine or packet processingengine 240. In brief overview, at step 310 of method 300, the device 104receives a network packet or is requested to transmit a network packet.At step 315, the device 104 requests the packet processing engine 240 toprocess the network packet responsive to the packet processing timer242. As part of, or associated with, packet processing operations, atstep 320, the packet processing engine 240 requests the cache manager232, policy engine 236, encryption engine 234, and/or the multi-protocolcompression engine 238 to perform an operation on a cached object. Atstep 325, the cache manager 232, policy engine 236, encryption engine234, and/or the multi-protocol compression engine 238 performs therequested operation, which may include any one or combination of thetechniques of the present invention described herein. In one embodiment,the cache manager 232 determines invalidation of a cached object, andmarks the cached object invalid. In some embodiments, the cache manager232 flushes the invalid object in response to a request by the packetprocessing engine 240. As the cache manager 232 is performing theseoperations responsive to the packet processing timer 242, invalidationof objects can occur within time periods in the order of millisecondsand with objects having an expiry in the order of the time intervalsprovided by the packet processing timer 242, such as 10 ms.

In further detail of method 300 of the present invention, at step 310,the appliance 104 receives one or more network packets, and/or transmitsone or more network packets. In some embodiments, the appliance 104requests to transmit one or more network packets over the network 108 ornetwork 110. In another embodiment, the appliance 104 receives a networkpacket on one port 266 and transmits a network packet on the same port266 or a different port 266′. In some embodiments, the packet engine 240of the appliance 104 transmits or requests to transmit one or morenetwork packets. In one embodiment, the appliance 104 receives ortransmits a packet on a first network 108, while in another embodiment,the appliance 104 receives or transmits a packet on a second network110. In other embodiments, the appliance 104 receives and transmitspackets on the same network. In some embodiments, the appliance 104receives and/or transmits networks packets to one or more clients 102a-102 n. In other embodiments, the appliance 104 receives and/ortransmits networks packets to one or more servers 106 a-106 n.

At step 315, the device 104 may request or trigger packet processingoperations of the packet processing engine 240 upon receipt of a networkpacket at the network port 266 of the device 104 or upon request totransmit a network packet from the device 104, or upon any combinationof receipt and/or transmit of one or more network packets. In someembodiments, the packet processing operations of the packet processingengine 240 are triggered via a signal provided by a packet processingtimer 242. In one embodiment, the packet processing timer 242 mayprovide interrupt-driven or event-driven timer functionality related tothe receipt and/or transmission of one or more network packets. In someembodiments, the packet processing timer 242 is driven by a rate ofreceipt and/or transmit of network packets via the device 104, or by therate by which each packet or a batch of packets are processed. As such,the packet processing timer 242 may be triggered and reset after eachset of one or more packet processing operations. In another embodiment,the packet processing timer 242 provides time intervals, either equal orvariable time intervals, to trigger, wake-up, or signal the packetprocessing engine 240 to perform a function or operation, such ashandling a received packet or transmitting a submitted packet. Asdiscussed above in connection with the device 104 of FIG. 2, the packetprocessing timer 242 may operate in the order of milliseconds, such ascausing time intervals or triggering of packet processing operations atintervals of 10 ms or less. The granular timer functionality of thepacket processing timer of the present invention may be provided invarious ways and used in operations of the packet processing operationsof the packet processing engine 240.

At step 320 of method 300 of the present invention, the packetprocessing engine 240 requests one or more of the cache manager 232,policy engine 236, encryption engine 234, and/or the multi-protocolcompression engine 238 to perform an operation. In one embodiment, thepacket processing engine 240 or packet processing timer 242 generates asignal or signals to one or more of the cache manager 232, policy engine236, encryption engine 234, and/or the multi-protocol compression engine238. The packet processing engine 240 may request or signal theoperation at any point before, during, or after a packet processingoperation of a network packet, or one or more packets. In oneembodiment, the packet processing engine 240 makes the request upontrigger of the packet processing timer 242 or expiration of a timeinterval provided by the packet processing timer 242, and beforeperforming a packet processing operation on a network packet. In anotherembodiment, during the course of performing one or more packetprocessing operations, the packet processing engine 240 makes therequest. For example, during execution of an operation, such as within afunction call, the packet processing engine 240 may make an applicationprogramming interface (API) call to one of the cache manager 232, policyengine 236, encryption engine 234, and/or the multi-protocol compressionengine 238. In other embodiments, the packet processing engine 240 makesthe request upon completion of a network packet processing operation.

At step 325, the requested operation is performed by one or more of thecache manager 232, policy engine 236, encryption engine 234, and/or themulti-protocol compression engine 238. In some embodiments, anyfunctionality or operation provided via the kernel 204 may be requestedto be executed, such as via a kernel application programming interface(API). As such, any of the functions of the device 104 may be performedin conjunction with the timing or timing intervals of packet processingvia the packet processing timer 232. In some embodiments, the requestedoperation is performed synchronously and in conjunction with the packetprocessing operations of the packet processing engine 240. For example,the packet processing operations wait and continue upon a completion of,or response from, the requested operation. In other embodiments, therequested operation is performed asynchronously with the packetprocessing operations. For example, the packet processing engine 240sends a request to perform the operation but does not block or wait toreceive a response from the operation. As will be discussed in furtherdetail in conjunction with method 350 of the present invention depictedin FIG. 3B, the packet processing engine 240 may request the cachemanager 232 to perform any cache management function, such as checkingfor expiry or invalidation of objects, marking objects as invalid, orflushing invalid or expired objects.

In some embodiments, the packet processing engine 240 at step 320 sendsmultiple requests, such as a first request to the cache manager 232 anda second request to the encryption engine 234. In other embodiments, thepacket processing engine 240, at step 320, sends a single requestcomprising multiple requests to be distributed by the device 104, suchas via the kernel 230 to the intended component of the device 104. Inone embodiment, the requests are communicated subsequent to each other.In another embodiment, requests may be dependent on the status, result,success, or completion of a previous request. For example a firstrequest to the policy engine 236 may be used to determine a policy forprocessing a network packet from another device or a user associatedwith the network packet. Based on a policy of the policy engine 236, asecond request to the cache may be made or not made depending on aresult of the first request. With the cache manager 232, policy engine236, encryption engine 234, and/or the multi-protocol compression engine238 integrated in the kernel space 204 of the device 104 with the packetprocessing engine 240, there are various operations of the device 104 asdescribed herein that may be triggered by and integrated with packetprocessing operations.

2. Invalidation Granularity

In another aspect, the present invention is related to and incorporatesthe ability to configure the expiration time of objects stored by thecache to fine granular time intervals, such as the granularity of timeintervals provided by the packet processing timer. This characteristicis referred to as “invalidation granularity.” As such, in oneembodiment, the present invention can cache objects with expiry timesdown to very small intervals of time. In other embodiments, the cachemanager responsive to a packet processing timer can also receive aninvalidation command to invalidate an object within a very short timeperiod of caching the object. By providing this fine granularity inexpiry time, the cache of the present invention can cache and serveobjects that frequently change, sometimes even many times within asecond. One technique is to leverage the packet processing timer used bythe device of the present invention that is able operate at timeincrements on the order of milliseconds to permit invalidation or expirygranularity down to 10 ms or less. Traditional caches, by contrast tothe present invention, are typically not capable of achieving expiry orinvalidation granularity of less than one second.

Referring now to FIG. 3B, an embodiment of a method 350 of the presentinvention is depicted for invalidating or expiring a cached objectresponsive to the packet processing timer 242 and/or packet processingengine 240. As such, in some embodiments of the present invention,cached objects can be invalidated or expired in the order ofmilliseconds, such as 10 ms or less. In overview, at step 355 of method350, the cache manager 232 receives a signal or request to perform anoperation via the packet processing engine 240 in response to the packetprocessing timer 242. At step 360, the cache manager 232 determines if acached object, such as a dynamically generated object, is invalid orexpired. At step 365, if the object is invalid, the cache manager 232marks the object as invalid, and at step 370, flushes the invalid objectfrom the cache manager 232.

In further detail of step 355, in some embodiments, the cache manager232 may be signaled or requested to perform a cache related operation atany point of time during network packet processing. In one embodiment,at step 355, the cache manager 232 receives an operation request priorto the processing of a network packet received or to be transmitted bythe device 104. In another embodiment, the cache manager 232 receives anoperation request upon the completion of processing of a network packet.For example, the packet processing engine 240 completes processing of anetwork packet, and before either waiting for the next time interval ofthe timer 242 or before processing the next packet, requests the cacheto perform an operation. In other embodiments, during an operation ofpacket processing, the packet processing engine 240 communicates anoperation request to the cache manager 232. In another embodiment, thecache manager 232 receives a signal, such as from the packet processingengine 240 or packet processing timer 242 to trigger the cache manager232 to perform an operation. In some embodiments, the signal indicatesto invalidate a cached object or to expire an expiry of a cached object.

In some embodiments, the cache manager 232 may receive a request toperform a cache operation from an entity external to the cache manager232, such as a request to invalidate an object communicated by a server106 a-106 n, and processed by the packet processing engine 240. In oneembodiment, the cache manager 232 may receive an invalidation requestwithin 10 ms or less of caching the object, while in another embodiment,as short as 5 ms, 2 ms or 1 ms. In other embodiments, the cache manager232 may perform a cache operation responsive to the operations orfunctionality of the cache manager 232, such as the expiration of atimer to cause an object to be invalidated or during the processing ofany cache command. In other embodiments, the cache manager 232 uses thepacket processing timer 242 of the device 104 to trigger cacheoperations. For example, the timer 242 may trigger or signal the cacheto check for invalidation or expiry of a cached object at any timeinterval capable of being set by the timer 242. In one embodiment, thetimer 242 may be set to trigger or signal the cache within 10 ms or lessof being set, or in another embodiment, as short as 5 ms, 2 ms, or 1 msof being set. In some embodiments, the originating server 106 a-106 nmay set the expiry time of the object. In other embodiments, theappliance 104 or client 102 a-102 n may set the expiry time of theobject.

At step 360, the cache manager 232 determines the invalidation or expiryof an object stored in cache. In some embodiments, an object in cache isinvalidated based on the expiration of a timer. In one embodiment, thecache manager 232 may issue an invalidation command on an object basedon the expiration of a timer. In another embodiment, the object storedin cache is automatically invalidated by the cache manager 232responsive to the expiration of a timer, such as a timer set with thepacket processing timer 242. In some embodiments, responsive to thepacket processing timer 242, the cache manager 232 checks for theexpiration of any timers for cached objects. In one embodiment, thecache manager 232 determines an object timer has expired, while inanother embodiment, the cache manager 232 determines the object timerhas not expired. In a further embodiment, the cache manager 232responsive to a second trigger or second timer interval of the packerprocessing timer 242 will check a second time if a previously checkedobject timer has expired.

In some embodiments, the cache manager 232 parses, interprets, accesses,reads or otherwise processes an invalidation command or request toidentify the object to invalidate in the cache. In one embodiment, anentity external to the cache manager 232 issues an invalidation commandto the cache manager 232 to invalidate the object. In anotherembodiment, the external entity may issue the invalidation commandresponsive to a packet processing timer 242. If the object is validand/or has not been invalidated, the cache manager 232 invalidates theobject responsive to the request. In some embodiments, the invalidationrequest processed by the cache manager 232 is responsive to the packetprocessing operations of the packet processing engine 240 processing therequest, which in turn may also be responsive to the packet processingtimer 242.

At step 365, the cache manager 232 marks the object as invalid. Thecache manager 232 may mark each object as invalid in any suitable ordesired manner. In one embodiment, an object is marked as invalid bysetting a flag, attribute, or property of the stored object. Forexample, a flag may be set to any value identifying to the cache manager232 the object is invalid. In another embodiment, an object may bemarked as invalid by moving the object to an area or portion of thecache for storing invalid objects. In other embodiments, the cachemanager 232 may identify or track the invalid and/or valid state of astored object by a database or a linked list or any type and form ofdata structure. In some embodiments, the cache manager 232 uses one ormore objects to identify or track the validity or invalidity of one ormore objects stored in cache. In another embodiment, the object ismarked as invalid by changing, modifying or altering the stored object,for example deleting or removing a portion of the object so that is maynot be used, or by changing or mangling the name of the object.

At step 370, the cache manager 232, in some embodiments, flushes fromthe cache those objects marked as invalid. In another embodiment, thecache manager 232 flushes the invalid object from cache upon request forthe object, such as by a client 102 a-102 n. In some embodiments, thecache manager 232 overwrites the invalid object with an updated copy orversion of the object received after invalidation or expiration of theobject. In another embodiment, the cache manager 232 reuses the cachememory occupied by the invalid object by storing another to the sameportion of cache memory. In yet another embodiment, the cache manager232 does not flush the object marked as invalid but keeps the objectstored in memory or storage of the cache.

Although method 350 describes invalidation and flushing of cachedobjects responsive to a packet processing timer and/or in conjunctionwith packet processing operations to provide invalidation granularity,any operation of the cache and any techniques of cache management aswell as any other operation of the device 104 described herein may beexecuted at fine granular time intervals provided by the packetprocessing timer. In some embodiments, the invalidation or expiration ofcached objects can occur as short as a 100 ms time interval, while inanother embodiment, as short as a 50 ms time interval. In someembodiments, the invalidation or expiration of cached objects can occuras short as 25 ms time interval, and in other embodiments, as short as a10 ms time interval. While in other embodiments, the invalidation orexpiration of cached objects can occur as short as a 5 ms time interval,and still yet in further embodiments, as short as a 3, 2, or 1 ms timeinterval.

By incorporating the capacity to invalidate objects after the elapse ofvery small increments of time as described in methods 300 and 350 inconjunction with FIGS. 3A and 3B above, improved caching of dynamicallygenerated content is enabled. Some dynamic content is in fact amenableto being stored and served from a cache for very short periods of time.To successfully cache such content, however, an approach in accordancewith an embodiment of the present invention provides caching objects forvery short periods of time before the object is invalidated and flushedfrom the cache memory. For example, certain dynamically generatedobjects may be cacheable for as long as 1 second but anything longer isfrequently unacceptable for content that is constantly changing. In anembodiment, the approach of the present invention included invalidatingor expiring cached content after small fractions of a second. As anexample, if an application 100 takes milliseconds to generate a dynamicresponse, then the cache can store and serve that response for aduration of less than or equal to the period of 100 milliseconds,without compromising the freshness of the data. There will not be a newobject generated during that 100 millisecond period because it isshorter than the time it takes to generate a new object. The appliance104 can thus be set up to serve the prior object during that duration.The ability of the appliance 104 to invalidate down to very smallincrements of time is frequently very useful for applicationenvironments where the database transaction isolation level is set toallow Repeatable Reads or Serialized Reads.

3. Invalidation Commands

Traditional caching technology invalidates stored content based on apre-defined expiry time for the content, which is typically configuredeither by the administrator or is received from the server that servedthe object. Described below is another technique of the presentinvention for invalidating content in order to more efficiently cachedynamically generated content. The technique of the present inventionincludes the ability to receive at the appliance 104 an invalidationcommand that identifies one or more of the previously stored objects inthe cache as invalid in real time. For example, the invalidation commandmay be communicated via a network packet transmitted to the client or anapplication programming interface (API) call made by a server to theappliance. This differs from the traditional approach by which theserver simply sets a cache expiry time that it includes in the objectheader at the time the object is served.

The technique of the present invention is more specifically illustratedin FIGS. 4A and 4B. FIG. 4A is a flow chart illustrating a method formaintaining a cache, such as a computer memory cache. In brief overviewand according to step 410, dynamically generated objects previouslyserved from an originating server 106 a-106 n are stored in the cache.For example, the dynamically generated object may not be identified ascacheable or otherwise include any cache or cache control information.At step 420, an invalidation command is received at the cache or cachemanager 232. The invalidation command identifies one or more previouslyserved objects as invalid. As step 430, in response to the invalidationcommand, the cache or cache manager 232 marks the identified object asinvalid.

In further detail at step 410, the cache manager 232 stores in a cachememory element a dynamically generated object received, obtained orcommunicate from any source. In some embodiments, the dynamicallygenerated object may be generated and served from a server 106 a-106 n.In other embodiments, the dynamically generated object may be generatedand communicated by a client 102 a-102 n. In some embodiments, anotherportion, component or process of the appliance 104 generates the objectand stores the object in the cache. In further embodiments, thedynamically generated object may be generated by another appliance 104or another computing device on the network and transmitted orcommunicated to the appliance 104. In some embodiments, the dynamicallygenerated object is not identified as cacheable or identified asnon-cacheable. In other embodiments, the dynamically generated object isidentified as cacheable or is under cache control.

At step 420, the cache manager 232 receives an invalidation commandidentifying an object to invalidate, such a dynamically generated objectstored in the cache. In one embodiment, the invalidation command maycomprise any type of directive or instruction indicating to the cachethat an object in invalid or otherwise may be stale. In someembodiments, the invalidation command identifies the object and may alsoidentify the time at which the object is invalid as well as whatportions of the object may be invalid. In one embodiment, the cachemanager 232 provides an application programming interface (API) that maybe called remotely by an originating server 106 a-106 n. In someembodiments, the cache manager 232 may provide any type and form ofprotocol for receiving commands and replying to commands via one or morenetwork packets. In one embodiment, the cache manager 232 or device 104provides an Extensible Markup Language (XML) API interface for receivingand processing invalidation commands. For example, the cache manager 232may provide a web service interface. In some embodiments, the cachemanager 232 replies to the invalidation command by sending anacknowledgement, status or other response to the originating server 106a-106 n. In other embodiments, the cache manager 232 does not reply tothe invalidation command. In one embodiment, an object is marked asinvalid if an application running in an originating server 106 a-106 nperformed an action that made the stored object stale, such as bygenerated a new or updated version of the object. This could occur, forexample, when news editors make changes to a fast developing news storyand therefore want to be assured the most recent version of the story isbeing served to clients.

Invalidation commands may be issued from an originating server by theapplication that generated the object, by another server 106 a-106 n oranother appliance 104. In one embodiment, the originating server 106a-106 n issues or communicates an invalidation command to the cache 232automatically in response to a change to the dynamically generatedobject on the originating server 106 a-106 n. The invalidation commandcan also be generated by an administrative control outside or externalto the server 106 a-106 n and the appliance 104. For example, theadministrative control may be any type and form of program orapplication running on the network and in communication with theappliance 104, such as administrator console. Furthermore, a client 102a-102 n could issue or communicate an invalidation command to theappliance 104 or cache manager 232. For example if the client were totake action that the client 102 a-102 n recognizes would cause a changeto the requested objects at the originating server, the client maycommunicate the invalidation command. Any object stored in the cache canbe invalidated by the transmission to the cache of a user commandexecuted locally at the cache or invoked remotely using the XML APIinfrastructure.

According to step 430, an object stored in cache, e.g., a previouslyserved dynamically generated object, that has been identified as invalidis marked as such in response to the invalidation command. An invalidobject will not be provided to a requesting client from the cache, butinstead would be served directly from the originating server. The cachemanager 232 may mark each object as invalid in any suitable or desiredmanner. In one embodiment, an object is marked as invalid by setting aflag, attribute, or property of the stored object. For example, a flagmay be set to any value identifying to the cache manager 232 the objectis invalid. In another embodiment, an object may be marked as invalid bymoving the object to an area or portion of the cache for storing invalidobjects. In other embodiments, the cache manager 232 may identify ortrack the invalid and/or valid state of a stored object by a database ora linked list or any type and form of data structure. In someembodiments, the cache manager 232 uses one or more objects to identifyor track the validity or invalidity of one or more objects stored incache. In another embodiment, the object is marked as invalid bychanging, modifying or altering the stored object, for example deletingor removing a portion of the object so that is may not be used, or bychanging or mangling the name of the object.

In some embodiments, the appliance 104 subsequently flushes from thecache those objects marked as invalid. In another embodiment, theappliance 104 flushes the invalid object from cache upon request for theobject, such as by a client 102 a-102 n. In some embodiments, theappliance 104 overwrites the invalid object with an updated copy orversion of the object. In another embodiment, the appliance 104 reusesthe cache memory occupied by the invalid object by storing anotherdynamically generated object to the same portion of cache memory.

With the command invalidation API of the cache manager 232 of thepresent invention, any computing device or user in communication withthe appliance 104 may request to invalidate an object, such as adynamically generated object, stored in the cache. As such, theinvalidation of objects stored in cache can be controlled real-timeinstead of using pre-determined configuration expiry or invalidationtime periods. Thus, using these techniques the longevity of the cachedobjects can be controlled from external application processing nodessuch as databases or originating application servers. For example, theappliance 104 can be configured to work with a database such that achange to the database automatically triggers an invalidation commandfrom the database (or application) to the appliance 104 to flush aparticular object or objects.

4. Invalidation of Groups Using Invalidation Command

In a further embodiment of the present invention, the appliance 104identifies and invalidates at the same time a group of objects stored bythe cache. Objects stored in a traditional cache memory are each treatedindividually and separately by the cache in determining whether theobject is stale. As each object reaches its specified expiry time(generally as set by the server and stored by the cache in a table) thatitem is flushed from cache memory. This traditional approach isinefficient and ultimately insufficient, however, to successfully handlethe challenges that arise in attempting to cache dynamically generatedcontent.

FIG. 4B illustrates another embodiment of a method of the presentinvention for maintaining a cache, such as a computer memory cache,wherein the appliance 104 has the ability to create, store, andinvalidate groups of related objects that have been previously servedfrom an originating server 106 a-106 n. In brief overview, at step 410,an object, such as a dynamically generated object served from anoriginating server 106 a-106 n is stored in the cache. At step 412, thecache manager 232 forms a group of previously served objects stored inthe cache. In one embodiment, the group may be associated with oridentified by one or more object determinants as will be described infurther detail below. At step 414, the cache manager 232 maintains arecord of the group of objects. At step 422, the cache manager 232receives an invalidation command to invalidate the group of objects. Atstep 432, the cache manager 232 marks the group of objects as invalid inresponse to the invalidation command.

Step 410 is the same as in FIG. 4A, wherein an object is stored in thecache of the appliance 104, such as dynamically generated objectspreviously served from an originating server 106 a-106 n. In someembodiments, one or more of the objects may not be identified ascacheable, or otherwise may not have any cache or cache controlinformation. For example, the server 106 a-106 n may assume thedynamically generated objects will not be cached.

According to step 412, the appliance 104 forms a group out of a set ofthe objects previously served from the originating server 106 a-106 nand stored in the cache. Any suitable or desired set of objects may beassociated with each other to form a group. For example, any dynamicallygenerated objects generated for, or associated with, serving a web pagemay form a group. In some embodiments, an object may be associated withmultiple groups. In other embodiments, one group of objects may form asubset of another groups of objects. In some embodiments, the formedgroup of objects have objects served from the same server 106 a-106 n,while in other embodiments, the formed group of objects have objectsserved from different servers 106 a-106 n. In further embodiments, theformed group of objects may comprise objects from a client 102 a-102 n,objects from a server 106 a-106 n, or objects generated by or servedfrom both clients 102 a-102 n and servers 106 a-106 n. In oneembodiment, one object in the group is static while another object inthe group is dynamically generated. In some cases, one object in thegroup is not identified as cacheable while another object in the groupis identified as cacheable. In other cases, the objects in the group maybe logically related in accordance with functionality or applicationprovided by a server 106 a-106 n. In another case, the objects in thegroup may be related as associated with the same client 102 a-102 n orthe same user.

In step 414, a record of the group of objects is maintained. Varioustechniques for recording and maintaining a record of a group of objects,or otherwise associating objects, may be used in practicing theoperations of the present invention described herein. In one embodiment,the record may be maintained directly in, for example, a look-up table.In another embodiments, the records could be represented in a hash-tableformat. In some embodiments, the cache manager 232 maintains theassociation of objects in a database, or a data structure or object inmemory. In further embodiments, a flag, property or attribute of eachobject in the group is assigned or set to a value identifying the group,such as a value equal to, identifying, or referencing the name oridentifier of the group, such as a group's object determinant that willbe described in more detail below. In some embodiments, a group ofobjects is arranged, placed or located in a portion of cache memoryidentified as holding the group

In step 422, an invalidation command is received at the appliance 104 orcache manager 232. According to the embodiment described in FIG. 4B, theinvalidation command identifies that one or more objects are invalid, orotherwise are stale. In some embodiments, the invalidation commandreferences, identifies or specifies a name or identifier of the group ofobjects. In one embodiment, the invalidation command comprises a singleinvalidation request to invalidate all the objects in the group. Inanother embodiment, the invalidation command identifies one object inthe group to invalidate. In other embodiments, the invalidation commandcomprises a plurality of invalidation request to invalidate a pluralityof objects in the group

According to step 432, the group of previously served objects is markedas invalid if the invalidation command references, identifies, orspecifies an object of the group as invalid, each object in the group asinvalid, or the group as invalid. In some embodiments, if theinvalidation command identifies an object in the group as invalid, thecache manager 232 marks the object as invalid. In other embodiments, ifthe invalidation command identifies an object in the group as invalid,the cache manager 232 marks the group of objects as invalid or eachobject in the group as invalid. In yet further embodiments, the cachemanager 232 may only invalidate the group of objects when a plurality ofobjects are identified as invalid via one or more invalidation commands.In another embodiment, the invalidation command may specify a name oridentifier of the group, and the cache manager 232 marks the group asinvalid, or each object in the group as invalid.

In one embodiment, the appliance 104 or cache manager 232 flushes fromthe cache memory a group of objects that has been marked as invalid. Insome embodiments, the objects in the group may be flushed from cachememory only when each object in the group is marked as invalid. In otherembodiments, if one object of the group has been marked as invalid thenthe entire group is flushed. In another embodiment, the group ofobjects, or any object in the group, marked as invalid may be flushedupon receipt of a request for the group of objects, or any object ingroup, by a client 102 a-102 n. In other embodiments, the group ofobjects, or any object in the group, marked as invalid may be flushedupon receipt of a response from a server 106 a-106 n provide one or morenew objects in the group.

An example of the above described embodiments follows. Customer resourcemanagement (“CRM”) applications are used by many businesses to track andevaluate all aspects of resource management. Often, CRM applications areimplemented and accessed over private and public networks including theInternet. These applications, which provide access to large amounts ofdata that is frequently being accessed, thus benefit from caching thedata generated by such applications. For example, sales reports arefrequently generated and served to remotely connected users. These salesreports are built by the relevant application through compiling datafrom sales information that is posted to such application servers and/ortheir underlying databases. As many users request the same document(i.e., a certain sales report), without caching, the application servermust re-generate the object for each request. If, however, such objectscan be stored in the cache, then application and database processing isconserved, including potentially valuable bandwidth, as the cache isplaced closer to the requesting clients.

The challenge for caching such objects arises because each time a newsale is posted to the application running at the originating server (orto its underlying database), the information in the sales report needsto be updated. As a result, all sales reports that may have been storedin any caches supporting these application servers must be invalidatedand the content flushed out of cache memory. The traditional approach tocaching, however, has no way of accurately determining when the changeto the underlying database or application is going to occur andtherefore cannot reasonably evaluate the freshness of dynamic content.Every time a change occurs in database or application or originatingserver, the cache has to be able to identify that the change has beenmade, and which group of objects should be invalidated as a consequenceof such change. Generation of invalidation commands that contain objectdeterminants linked to groups of previously served objects, as describedabove, can meet this need.

Multiple groups of related objects may be formed at a singlehierarchical level. Alternatively, sub-groups of objects may be formedto create multiple hierarchical levels. In an embodiment, the groups orsub-groups of objects may be pre-designated by a user. In anotherembodiment, a user may establish rules by which the appliance 104automatically forms groups of related objects, and associates objectdeterminants therewith.

5. Identification of Object Determinants in a Client Request or Response

An embodiment of the present invention also addresses the need to beable to identify all objects affected by a state change at theoriginating application server 106 a-106 n (and/or underlying database)by generating groupings of objects and implementing parameterizedinvalidation. In this embodiment, any object or pre-defined group ofobjects can be invalidated by an intercepted HTTP request, for examplefrom a client, that the cache parses in order to identify an objectdeterminant. The term “object determinant” refers to any information,data, data structure, parameter, value, data pattern, request, reply, orcommand that references, identifies or specifies one object or a set ofobjects, uniquely or otherwise. In some embodiments, an objectdetermination is a pattern of bytes or characters in a communicationthat may be associated with an object or used to uniquely identify thatthe communication is associated with, or referencing, the object. In oneembodiment, an object determinant indicates whether change has occurredor will occur, in the originating server, to a group of previouslyserved objects stored in the cache manager 232 with which the objectdeterminant is associated. In some embodiments, the objects in a groupof objects are related in that they are associated with at least oneobject determinant. Specific, non-limiting examples of objectdeterminants and further illustrations of their use are described morefully below.

In some embodiments of the present embodiment, object determinants arecertain pre-defined parameters or data structures included or embeddedin a client request or response. In other embodiments, the client 102a-102 n, server 106 a-106 n or appliance 104 embeds in a communicationone or more object determinants, such as pre-defined strings or sets ofcharacters representing the object determinant. The object determinantsindicate whether such request will have the effect of causing a changein the state of objects stored in the originating server 106 a-106 n ordatabases linked thereto. In one embodiment, the existence of the objectdeterminant in a request indicates a change has or will occur to anobject. In another embodiment, the syntax, structure, parameter, orvalue of the object determinant indicates a change has or will occur toan object. In an embodiment, the cache receives an object request from aclient 102 a-102 n. The request may include certain parameters or values(object determinants) that the cache recognizes will change the state ofthe originating server or application server which will, as aconsequence, make stale certain related objects stored by the cachemanager 232 that had been previously generated by such originatingserver or application server 106 a-106 n. Depending on the invalidationpolicy set by the user, the parameters (object determinants) may requireinvalidation of one or more previously served objects or group ofobjects, by the originating server, that have been stored by the cache.The cache is configured to identify the relevant objects that will beeffected by this state change (i.e., those objects or groups of objectslinked to the object determinant), and invalidate these objects via themethod marking each of the objects as invalid and/or flushing suchobjects from the cache memory.

The above described technique is illustrated in FIG. 4C. As with otherembodiments described herein, step 410 comprises storing, in the cache,objects, such as dynamically generated objects previously served from anoriginating server. The objects could be generated by an applicationrunning on the originating server 106 a-106 n, or could be drawn, forexample, from a database accessed by the originating server 106 a-106 n.In some embodiments, the dynamically generated objects are identified asnot cacheable or otherwise not identified as cacheable.

According to step 421, the cache intercepts or otherwise receives acommunication between the client and the server, such as a request froma client or a response from a server. In some embodiment, the request isfor a specified object, the object having been previously served andstored in the cache. In another embodiment, the communication includes aresponse from a server having a requested object. In one embodiment,such receipt or interception occurs according to established cachingprotocol and communications standards. Although the cache manager 232 orappliance 104 may be generally described as receiving a request,response or communication, in receiving such request, response orcommunication, the cache 232 or appliance 104 may intercept or obtain byany suitable means and/or mechanisms the request, response orcommunication even though not communicated directly or explicitly to thecache.

In step 423, an object determinant is identified in the interceptedcommunication. The cache manager 232 may extract, interpret, parse,access, read, or otherwise process the intercepted communication todetermine or identify one or more objects determinants in thecommunications. Any parameter, value, syntax, data, structure or set ofone or more characters of the communication may be used to identify anobject determinant. In one embodiment, the cache manager 232 mayidentify the name or identifier of an object in a request from theclient 102 a-102 n to the server 106 a-106 n, in which the clientrequests the object. In another embodiment, the cache manager 232 mayidentify the name or identifier of a first object in the request of theclient 102 a-102 n or response from the server 106 a-106 n thatindicates a change has occurred or will occur to a second object storedin the cache. In other embodiments, the cache manager 232 determines ifany patterns of characters in the request match any object determinantsassociated with an object or group of objects in the cache. In someembodiments, an object determinant may be determined for an object notcurrently stored in cache. In other embodiments, an object determinantmay be determined for an object currently marked as invalid. In otherembodiments, an object determinant for a requested object is determinedto be associated with an object determinant of a cached object. In yetanother embodiment, upon the first reference, request, or response foran object in a communication, the cache manager 232 establishes theidentified object determinant as the object determinant for the object.

By receiving and parsing the communication, such as a client request orserver response, to identify an object determinant, the cache manager232 or appliance 104 may effectively determine whether to mark asinvalid a cached object that has been associated with the identifiedobject determinant. Thus, according to step 425, a determination is madeas to whether the object determinant indicates a change to the cachedobject. In some embodiments, the identified object determinant may bepart of a communication that does not alter, modify or generate anobject. In other embodiments, the identified object determinant is apart of a communication that indicates a change has occurred or willoccur to the object associated with the object determinant. For example,the communication may be a get request for a dynamically generatedobject or a submit request that will change the data used for one ormore dynamically generated objects. In some embodiments, the existenceof the object determinant in the communication indicates a change has orwill occur on one or more objects. In another embodiment, the type orname of a command, directive or instruction in the communication alongwith the object determinant indicates a change has or will occur on oneor more objects. In yet a further embodiment, the existence, value orsetting of a parameter or variable of a command, directive orinstruction indicates a change has or will occur on one or more objectsassociated with an object determinant.

In other embodiments, the cache manager 232 performs a hash function,algorithm, or operation on the intercepted communication or objectdeterminant to determine if a change has occurred in the object. In someembodiments, the hash value is compared with a previous stored hashvalue for the object and if different then the cache manager 232recognizes the object has changed. In yet another embodiment, a hashvalue for the object may be included in the communication or objectdeterminant. In one embodiment, the communication indicates the objecthas changed by the value or setting of a parameter, such as with aBoolean flag. In other embodiments, an entity tag control and validationmechanism as will be described in more detail below may be used toidentify the object and determine if the object has changed.

If a change is indicated, then at step 431, then the object associatedwith or identified by the object determinant is marked as invalid. Insome embodiments, an object requested by the intercepted communicationis marked as invalid in accordance with step 431, and retrieved from theoriginating server 106 a-106 n in accordance with step 440. Otherwise,in other embodiments, the requested object is retrieved from the cachein accordance with step 450. In one embodiment, any object marked asinvalid will be flushed from the cache.

6. Invalidation of Groups of Objects Based on Object Determinants

The above embodiment of the present invention describes the case ofinvalidating a previously served object in the cache manager 232 basedon identification of an object determinant in the client request. Thisgeneral concept may also be used, in another embodiment, to identify andinvalidate a group of objects with which one or more object determinantshave been associated. This embodiment is illustrated in FIG. 4D.

The method described in FIG. 4D begins in the same fashion as the methodof FIG. 4C. Step 410 comprises storing, in the cache, objects, such asdynamically generated objects previously served from an originatingserver. In some embodiments, one or more of the objects are notidentified as cacheable. According to step 412 and similar to FIG. 4B,previously served objects are formed into groups. In one embodiment andin accordance with the object determinant technique of the presentinvention, a group of objects is associated with or identified by atleast one object determinant. As described more fully below, in someembodiments, the association of groups with object determinants dependson the nature and details of the users caching policy, such as a policydefined, controlled or used by the policy engine 236. In otherembodiment, the one or more object determinant of the group comprisesthe one or more object determinants of the objects in the group. Inanother embodiment, the object determinant of the group comprises acombination of object determinants of objects in the group.

According to step 414, a record is maintained of the group, along withits associated object determinants, if applicable. This step is similarto step 414, illustrated in FIG. 4B. In one embodiment, the recordand/or any object determinants of the group is maintained in a look-uptable. In other embodiments, the record and/or any object determinantsof the group may be maintained in a hash-table format. The hash-tablemay be designed to efficiently store non-contiguous keys that may havewide gaps in their alphabetic and numeric sequences. In anotherembodiment, an indexing system can be built on top of a hash-table. Insome embodiments, the cache manager 232 maintains the association ofobjects as a group with one or more object determinants in a database,or a data structure or object in memory. In further embodiments, a flag,property or attribute of each object in the group is assigned or set toa value identifying the group, such as a value equal to, identifying, orreferencing the name or identifier of the group, or a group's objectdeterminant. In some embodiments, a group of objects is arranged, placedor located in a portion of cache memory identified as holding the group.In another embodiment, the one or more object determinants are stored inassociation with the group of objects.

Steps 421 and 423 are similar to steps 421 and 423 as illustrated inFIG. 4C. According to step 421, the cache manager 232 or appliance 104intercepts or otherwise receives a communication between the client 102a-102 n and server 106 a-106 n, such as a request from a client for anobject previously served and stored in the cache. In one embodiment, thecache manager 232 intercepts a request from the client 102 a-102 n tothe server 106 a-106 n. In some embodiments, the request is for anobject stored in cache. In other embodiments, the request is aninstruction, command or directive to the server 106 a-106 n that willcause a change to an object stored in cache, such as to cause an objectto be dynamically generated. In another embodiment, the cache manager232 intercepts a response from a server 106 a-106 n to the client 102a-102 n comprising or identifying an object stored in cache.

In step 423, an object determinant is identified in the interceptedcommunication. As noted above, the object determinant indicates whethera change has occurred or will occur in the requested object, at theoriginating server 106 a-106 n. However, in the embodiment of FIG. 4D,the object determinant may be associated with a group of objects. Thisenables efficient invalidation of all objects stored in the cache thatmay be affected by a particular object determinant. In some embodiments,an object determinant of an object in the group is identified. In otherembodiments, an object determinant, for example, a group objectdeterminant, for the group of objects is identified. In anotherembodiment, a combination of object determinants of one or more objectsin the group are identified.

Thus, according to step 427, a determination is made as to whether theobject determinant indicates a change in the group of previously servedobjects. In some embodiments, the existence of the object determinant ofthe group in the intercepted communication indicates a change hasoccurred or will occur to one or more, or all of the objects in thegroup. In other embodiments, the name and type of a command, directiveor instruction in the intercepted communication indicates such changes.In yet another embodiment, the existence, value or setting of anyparameters or variables in the communication may also indicate suchchanges.

If at step 427, the object determinant indicates a change in the group,then the group of previously served objects is marked as invalid in thecache in accordance with step 435. In some embodiments, one or more, orall of the objects of the group are requested and retrieved from theoriginating server 106 a-106 n in accordance with step 440. If at step427, the object determinant does not indicate a change in the group,then in some embodiments, any objects requested as part of interceptedcommunication and previously served and stored in the cache is retrievedfrom the cache manager 232 in accordance with step 450. In anembodiment, any object or group of objects marked as invalid may beflushed by the cache manager 232 from the cache.

7. Designation of Groups

The cache administrator may specifically designate which objects getincluded into a particular group. Whenever an object is stored in thecache, the administrator may make that object a member of one of theconfigured or implicit groups depending on the configuration. Theconfigured groups can be based on configurations that an administratorhas previously established or alternatively based on applicationbehavior and other data related to object invalidation. An object mayalso be part of an implicit group if its configured group is dynamic.Objects in the implicit group are grouped by the value of thesignificant invalidation parameters.

By permitting very flexible grouping of objects, a cache can achieve alevel of flexibility and coordination in invalidation that is necessaryto effectively cache dynamically generated content. The cache caninvalidate a very specific group of objects simultaneously, therebymaking the cache more responsive to the frequent need to invalidatedynamically generated content. At the time the cache assigns an objectto a group, the group determines a number of things relative to thatobject, including the invalidation parameters and the hit determinants,in order to associate one or more object determinants therewith.

In the customer resource management (“CRM”) example, the cacheadministrator may pre-designate each of the groupings. For example, theadministrator configures the cache to group each of the salesdepartments by name. Thus the administrator can designate an autodepartment, a motorcycle department, etc., and each time an objectdeterminant is recognized in a request coming to the cache, the cachecan then invalidate all objects stored in a designated group linked toan appropriate department via the object determinant.

8. Ruled-Based Grouping

Alternatively, the cache administrator may establish rules that allowthe cache appliance to determine on the run which objects to include ina particular group or groups. Such rules-based groupings may rely on thedesignation of groups by virtue of established rules that link theobject to significant object determinants that the cache utilizes tocreate the relevant groups. An example of this approach may involveconfiguring the cache with rules that the cache uses to recognize whatobjects to put in each group.

Again turning to the CRM example, a rule may state that each subdivisionof the Sales Department that is set up on the application should berecognized by the cache as its own grouping. In this way the groupingscan be created without the cache administrator having to specificallyidentify each grouping but allows the cache to determine based on therelevant rules. This technique creates a more flexible and often lesswork intensive way to designate groupings. The cache administrator couldconfigure a rule that states that every subdivision department of Sales(i.e., sales\auto, sales\motorcycle etc.) should generated a newgrouping by the cache. As a request from the Auto Sales Department isprocessed and returned by the application via the cache, the cache canrecognize each subgrouping of sales and automatically create a groupingfor it, based on the pre-configured rule.

The rule may be implemented by the cache each time it sees a new requestfor an object of the type report/sales/auto or report/sales/motorcycle,etc. This process can then be repeated when a Motorcycle SalesDepartment request showing that it is a sub-grouping of the SalesDepartment, then the Bicycle Sales Department and so forth, as the cacherecognizes these subgroups and establishes an object grouping for eachof them. When a known invalidation request comes to the cache linked toone of these groupings, or if a relevant object determinant isidentified in a client request (for example a post of a sales report tothe Motorcycle Sales Department sales/motorcycle found in the parsingthe request), the cache knows to invalidate all the cached objects inthe Motorcycle Sales Department Grouping.

In this way, when a cache recognizes that a change has occurred or willoccur to data served by the application (either because the cacherecognizes that contents of a request received by the cache will triggera change at the application or because of the occurrence of some outsidechange), the above technique enables the cache to quickly and simplyidentify which objects require invalidation through the process ofgrouping. In this way, the cache is able to invalidate large numbers ofdynamically generated objects that are no longer fresh because ofchanges in the application or database state.

The ability of the cache to successfully store and serve out of itscache memory dynamically generated content can also be enhanced with anintelligent statistical engine that examines the pattern of request andresponse traffic in order to determine, over a period of time, the setof objects that would provide the most caching benefit. The engine caneither be integrated into the cache appliance itself, or run in aseparate computer as a heuristic to select some subset of objects forfurther investigation to determine suitability for dynamic caching.

9. Further Use of Object Determinants

As described above, object determinants may be any data structure thatindicates whether a change has occurred or will occur, in theoriginating server, to the group of previously served objects stored inthe cache with which the object determinant is associated. Objectdeterminants could be set up on the basis of predefined string valuesembedded in the request. For example, when a request comes in with acertain USERID, the USERID can be linked to a group of objects in thecache memory that should be invalidated each time a post or otherrequest comes from that certain USERID. Potential candidates for objectdeterminants could also include using service identifiers of the serverthat originally served the object. The service identifier containsservice IP address, TCP port and service identifier present in the HTTPrequest.

Another potential object determinant present in the request the requestuniform resource locator (“URL”). In the case of caching of staticobjects, the request URL is typically sufficient to uniquely identifythe object. For requests for dynamically generated content, however, theinformation present in the URL may not be sufficient to identify thecached object. The cache must therefore inspect other information in therequest to find object determinants including in HTTP headers, cookieheader or in other custom HTTP headers. The cache can additionally lookfor a subset of relevant parameter information in a variety of otherplaces in the client request, including, without limitation: in the URLquery string, in the POST body, in a cookie header, or in any otherrequest or response headers.

The problem in parsing a URL for object determinants is that the URL andother headers may contain a lot of information in addition to what isrelevant for the cache's decision. The cache must therefore be able toparse through quite a lot of information to be able to identify theappropriate object determinants. In addition, the data in the header isoften arbitrarily ordered, meaning there are no standardized ways thatsuch data is placed into the HTTP header and therefore a simplecomparison is often insufficient to locate the relevant objectdeterminants in such string.

If there is no pre-configured policy to match a particular objectdeterminant to a relevant object or group of objects stored in cachememory, the cache may still, in another embodiment, make such adetermination. For example, the cache may examine and parse variousaspects of the request to discover whether any other object determinantsmay be found in such request and used to link such request to particularobjects stored in the cache memory that should be invalidated.Alternatively, one could also enable the cache to examine a request forcertain object determinants that the cache determines, based on certainpre-defined heuristics, may meaningfully linked to particular objects orgroup of objects. For example, when the request comes into the cache foran update of a calendar associated with a particular USERID, anembodiment of the present invention could be set up to recognize thatall cached objects with USERID equal to the USERID of the requestupdating the calendar, and that contains the user's calendar for any oneparticular day, will need to be invalidated.

The cache may also assume that the object determinants are present as agroup of name=value or similar pairs in a non-specified order in the URLStem, in the queries present in the URL, in the POST body or in a Cookieheader. In an embodiment, it is assumed that the query is formatted as alist of name=value pairs. The user can therefore configure whichparameter names are significant. Every cached object is keyed usingfirst its access URL. The URL may look like/site/application/special/file.ext?p1=v1&p2=v2&p3=v3. The/site/application/special/file.ext part is the URL stem. Thep1=v1&p2=v2&p3=v3 part is the URL query and contains parameter-valuepairs. These parameter-value pairs may also be present in the POST bodyor in the Cookie headers.

In an embodiment, the user or administrator establishes that p1 and p2shall be the invalidation parameters or object determinants. The cachewill thereafter automatically group objects that have matching p1 and p2values. One way of implementing this grouping is to map p1 and p2 toprimary keys in database tables, i.e., to uniquely identifiable objectsin the table that the cache will know how to reference in order todetermine validation status. To update something in those databasetables, in order to reflect the fact that data stored in the cache is nolonger valid, the cache will specify new values for p1 and p2 and whenthe cache recognizes such new values the next time it goes to serve suchcontent, it will know to invalidate the linked objects stored in itsmemory. The cache, when it encounters such a request, on seeing theupdate request knows that it has to invalidate the group with matchingp1 and p2 values—because the cache understands that data in the originwill change, thereby affecting all objects that are related to those p1and p2 object determinants.

To address the more complex case where the administrator has notpre-configured specific parameters embedded in the request as objectdeterminants, the cache can deploy user-configured policies to extractthe relevant object determinants from the request to assist inidentifying when to invalidate groupings of objects. The determinantstring is then used to locate the group of objects stored in the cacheand invalidate such objects. These object determinants can be used toconfigure the cache to generate lists of significant parameter values.If an incoming write-request has matching values for the significantparameters then the objects tied to those parameter names should beinvalidated. Alternatively, a user could specify the policy frameworkaction that can extract the object determinant string from the request.The object determinant string is extracted from the write-request andall objects with matching determinant strings are invalidated. In thisalternative approach, a request arrives at the cache, the cache makes adetermination whether the request string matches an invalidation policy.The invalidation policy specifies objects in which content group shouldbe invalidated.

Alternatively, the cache could use any other user information that maybe present in the client request. As noted above, the authentication andauthorization integration allows the cache access to the userinformation. The USERID or the GROUPID could be one of the determinantsin the event the relevant grouping of cached objects are linked to auser or a group of users. Although user information is often animportant object determinant, the user information often may not bepresent in the HTTP request. In a further embodiment of the presentinvention, the dynamic caching aspects of the invention can be combinedwith another of Applicant's patent applications. To accomplish this,reference is made to Applicant's above referenced copending patentapplication Ser. No. 11/169,002 (“the Integrated Caching patent”). Thatapplication describes a system and method for integrating the cache witha variety of other networking elements including the ability to performcertain kinds of authentication, access control and audit (AAA)infrastructure. Thus, the level of security accorded to data that isgenerated by the applications is applied to data that is instead servedfrom a cache. This technique allows the applications to cache sensitive,access controlled information that could not otherwise be cached.

This approach allows the cache to identify users that do not includeidentifiable user information in the HTTP request but that may beidentifiable via the AAA approach described in the Integrated Cachingpatent. Such an approach enables the cache to identify the relevant userto a particular request through examining the authorization stateinformation that can be shared from the AAA processing. In a furtherembodiment, the integration enables the application of security policiesto information stored in the cache to prevent unauthorized users fromaccessing information stored at the cache.

This approach also address the challenge posed by the fact that asignificant portion of dynamically generated data requires that theclient requesting such data be authorized and authenticated before thecache can respond to the relevant request from the client. The cachemust have the ability to authorize requests made by authenticated usersso that applications can cache access-controlled objects and byintegrating such dynamic caching technology with authentication andauthorization information, this security can be achieved. The USERID orthe GROUPID will be one of the object determinants if the objects arepersonalized to a user or a group of users. Thus, the level of securityaccorded to data that is generated by the applications is applied tocached information as well. This technique allows the applications tocache sensitive, access controlled information that could not otherwisebe cached.

Finally, other information like time of day, state of the database atthe origin, etc., may be parsed from the request and used as objectdeterminants to determine whether objects stored in the cache are stillvalid. The cache may take care of this situation by configuringappropriate expiration behavior in groupings of objects that areconfigured to be sensitive to such external variables.

To further address the challenge presented by the fact that requests fordynamic content must be parsed and interpreted by the cache, the cachein accordance with an embodiment of the present invention can limitwhich parameters are deemed to be relevant object determinants for thecache. In this way, the success rate for serving objects from the cacherather than forwarding such requests to the applicable applicationserver can be enhanced. By way of example, a request query from a clientmay contain both a city and a state parameter. However, the cache may beconfigured to comply with the requirements of the application for whichthe cache is storing content to recognize that the response can beserved to requests coming from clients that the query shows come fromall clients in a given state without regard to the city value. For thispurpose, the city parameter is not relevant and the cache couldrecognize this fact. An alternate embodiment involves configuring thecache so that a response can be served from the cache if just the cityparameter makes a match regardless of what is specified for the stateparameter.

In summary, the cache implements generalized parameterized objectmatching. In this approach, the cache is configured to recognize thesubset of information in the request that will be useful as objectdeterminants, and that are linked to a particular object so that whensuch object determinants are recognized, the cache can utilize thepresence (or conversely the absence of such determinants) in evaluatingwhether the object or group of objects remains fresh and capable ofbeing served from the cache. The cache maintains a table that itconsults each time a request comes in to check against the configuredparameters to determine if the requested data remains fresh, and whichalso allows the cache to match the relevant data to the proper objectstored in the cache memory.

10. Incarnation Numbers

In yet another embodiment, the cache can utilize incarnation numbers toinvalidate a group of objects. Where a cache needs to change the stateof each of a group of objects at one time because of a change in thestate at the origin, incarnation numbers provides a simple technique foreffecting this invalidation. Whereas identifying each object andchanging the state individually is an inefficient approach to assuringfreshness of data stored in a cache, use of incarnation numbers enablesa much more simple and effective approach to invalidating groups ofobjects. The present embodiment describes how each object points to adata structure that represents the group and therefore the server needonly send a command that changes the state in the data structure for thegroup. When a subsequent request for a cached object arrives from aclient, the cache must first figure out whether the state has changed.To do so it looks up the data structure to reference whether the statehas changed for the group.

In order to implement the data structure effectively, the cache must beable to determine whether to look up for a state change. Therefore, thecache must be able to determine whether it has already looked at thestate change in the group or not. This is where the incarnation numbersare helpful. The cache associates dynamically generated objects intocontent groups. Each of these content groups may be represented througha hash table look-up process with a particular index value or“incarnation number” contained in a data structure. Thereafter, wheneverthe cache receives a client request that the cache recognizes as causinga state change, the client parses the client request for the relevantparameters, performs the hash look-up based on the recognized objectdeterminants, and increments the index or incarnation number in the datastructure. Each time an object stored within a designated grouping isrequested by a client, the cache performs the hash algorithm on theobject, and compares it to the original stored value in the datastructure for such content group. If the stored value is the same as thenumber calculated by the cache for such object, then the cache knows thecontent remains fresh and can be served to the requestor. In the eventthe cache detects a discrepancy between the current incarnation numbercalculated for such object in and the number stored for such contentgroup in the data structure, the cache knows that the stored object isno longer fresh. The cache then invalidates the stored object and sendsthe request along to the application server. When the response comesback the cache appliance will store the new response in the cache memoryand link such response again to the new data structure. Thereafter, eachtime the cache receives a request for an object in that grouping, thecache can make the comparison and assuming no further changes have beenmade to the data structure, the cache can serve the newly stored object.

By utilizing invalidation of a group of objects in this fashion, thecache is able to invalidate very quickly—and the time taken is constantregardless of the number of objects invalidated. Through this faster andmore efficient process of invalidation, the techniques of the presentinvention enables the cache to more effectively handle dynamicallygenerated objects. The approach allows cache appliances that sit infront of applications to more aggressively store and serve dynamicallygenerated objects without serving invalid or stale content because ofrapid changes in such data. The embodiment enables the cache to servedata that frequently or unpredictably changes thereby improving theperformance of the cache. The cache is also able to invalidate objectsand group of objects stored in the cache memory using user commands andalso by examining and grouping various kinds of web traffic.

11. Flash Cache and Flash Crowds.

Another embodiment of the present invention may also include a techniquethat is able to increase cache hit rates for extremely fast changingdynamically generated objects that would not otherwise be cacheable.When a cache manager 232 receives a first request from a client for aparticular object, the cache manager 232 forwards that request to theoriginating server 106 a-106 n for processing because, as the first suchrequest, the cache manager 232 cannot yet have such object stored. Asthe response object is generated and then returned to the requestingclient via the cache manager 232, the cache manager 232 automaticallystores a copy of the object. With objects that change extremely quickly,traditional caches, in contrast to the present invention, just pass allresponses along to the originating server 106 a-106 n for processing assuch content is always assumed to be not fresh, and therefore not validfor cache storage.

In one aspect, the present invention is directed towards a “flash cache”technique for handling additional requests for an object received by thecache manager 232 during the time the cache manager 232 or appliance 104is in the processing of transmitting or waiting to transmit a responsefor a first requestor of the object. Referring now to FIG. 5, method 500of the present invention depicts the flash cache technique of thepresent invention. In brief overview and in view of FIGS. 1 and 2, atstep 510, the cache manager 232 of the present invention receives aresponse from an originating server 106 a-106 n for an object requestedby a first client, for example client 102 a illustrated in FIG. 1. Theobject may comprise a dynamically generated object which is generated byone of the originating servers 106 a-106 n, and the response from theoriginating server 106 a-106 n may include the dynamically generatedobject. At step 515, the cache manager 232 requests transmission of theresponse to the first client, such as via a network stack, for example,a TCP/IP stack of the device 104. At step 520, the response may bestored or held in a buffer while waiting to be transmitted. For example,the device 104 may throttle communications to a slow connected or lowbandwidth client 102 a, and therefore, the device 104 queues networkpackets, which represent the response.

At step 525, while the response of the first client 102 a is waiting tobe transmitted in the buffer or otherwise in the process of beingtransmitted and/or prior to completing transmission of the response tothe first client 102 a, the cache manager 232 receives a second requestfor the object from a second client 102B. At step 530, the cache manager232 determines the object is currently in the buffer and provides theobject from the buffer to response to the second request to the secondclient 102B. In one embodiment, as the device uses a single TCP/IPstack, the same object can be provided for transmission to the firstclient 102 a and the second client 102B. At step 535, the first responseis transmitted to the first client 102 a, and a response is transmittedto the second client 102B. At step 540, the response of the first client102B is removed from the buffer.

In further detail, at step 510, the device 104, such as by the cachemanager 232, intercepts or otherwise receives a response from anoriginating server 106 a-106 n to a request from a first client 102 afor an object, such as dynamically generated object. In one embodiment,the dynamically generated object is identified as non-cacheable, such asby the originating server 106 a-106 n, or is otherwise not identified ascacheable. In some embodiments, the cache manager 232 receives therequest from the first client 102 a and forwards the request to theoriginating server 106 a-106 n. In one embodiment, the cache manager 232checks the cache manager 232 for the requested object prior toforwarding the request to the originating server 106 a-106 n. In somecases, the cache manager 232 determines the object is invalid or hasexpired. In other cases, the cache manager 232 determines the object isnot stored or otherwise available in the cache manager 232. For example,the object may have been flushed for example, or this may be the firsttime the object has been requested from an originating server 106 a-106n. In some embodiments, the response comprises the object, and in otherembodiments, the response indicates a status, such as a failure or errormessage regarding the object.

At step 515, the response for the request by the first client 102 a isprovided for transmission to the first client 102 a. In someembodiments, the device 104 receives the response, and requests thepacket processing engine 240 to transmit the response to the client 102a. In other embodiments, the cache manager 232 receives the response andrequests the packet processing engine 240 to transmit the response tothe client 102 a. In one embodiment, the device 104 comprises onenetwork stack for receiving and transmitting network packets to theclients 102 a-102 n. In some embodiments, the network stacks comprises aTCP/IP stack. In other embodiments, the device 104 may include multiplenetwork stacks, each associated with or used by one or more clients 102a-102 n. The network stacks or stacks are associated with the kernel204, network ports 226 and/or the packet processing engine 240 as thoseordinarily skilled in the art will recognize and appreciate.

At step 520, during the course of transmitting the response to the firstclient 102 a requested at step 515, the device 104 queues, holds, orotherwise stores the response in a buffer. In some embodiments, thebuffer is a part of the network stack, such as a TCP/IP stack. Forexample, the buffer may comprise a data structure used in or by theTCP/IP stack or by a network driver, filter or other network relatedsoftware processing or handling any portion of the network stack. Inother embodiments, the buffer is part of the network port 226. Inadditional embodiments, the buffer is part of the packet processingengine 240, or yet in further embodiments, the kernel comprises thebuffer. The buffer may be any memory or storage element located in anyportion of the device 105. In many embodiments, the response is queued,held or stored in the buffer as one or more network packets, such asTCP/IP protocol based packets. In one embodiment, the response is heldin one or more data structures providing the buffer for use by thepresent invention. As those ordinarily skilled in the art will recognizeand appreciate, the response may be queued, held or stored in the bufferin any suitable form.

Furthermore, the response may be stored or queued in the buffer for anyduration of time period, either arbitrary, predetermined, implicit,explicit or otherwise, and using any suitable means and/or mechanisms.In one embodiment, the response is stored in the buffer as the networkpacket awaits to be transmitted by the device 104 based on a rate oftransmission, packet queuing rate, network traffic and congestion, orany other network characteristics and as may be determined by the device104, such as via the packet processing engine 240 and/or the networkports 226. In another embodiment, the response is held in a buffer for adetermined time period. In some embodiments, the device 104 queues,manages and transmits network packets to a client 102 a based oncharacteristics, such as bandwidth, type of connection, etc. of theclient's network connection(s), or characteristics of the client 102 a.For example, for a client 102 a with a slower connection to the device104 than the device's connection to an originating server 106 a-106 n,the device 104 may throttle or otherwise manage the transmission of thenetwork packets to the client 102 a to provide a desired performance ofor behavior for the client 102 a, or the network connection of theclient 102 a. As such, the device 104 may queue or hold the response ina buffer for a desired time period in accordance with any queuing orbuffer management logic, function, rule or operation.

At step 525, while the response of the first client 102 a is held in thebuffer, or is being transmitted or otherwise prior to completingtransmission of the response to the first client 102 a, such as a bufferof a TCP/IP stack, the device 104, such as via the cache manager 232,intercepts or otherwise receives a second request from a second client102B for the object obtained by the cache manager 232 for the firstrequest. That is, in one embodiment, the object requested by the secondclient 102B is stored in the buffer waiting to be transmitted. In someembodiments, the device 104 receives multiple requests from multipleclients 102 b-102 n for the same object requested by the first client102 a and is currently being held in the buffer for transmission to thefirst client 102 a or is other currently being transmitted to the firstclient 102 a. In some embodiments, the cache manager 232 determines thesecond request is requesting the same object as the first request usingany suitable means and/or mechanisms, such as any of the techniquesdescribed herein for example, by using object determinants. In someembodiments, the appliance 104 or the cache manager 232 queues thesecond request, and any additional requests in a queue using any typeand form of queuing mechanism. As such, in one embodiment, the clients102 a-102 n do not need to resubmit the requests as the appliance 104 orthe cache manager 232 queues and responds to the requests on behalf ofthe servers 106 a-106 n. In another embodiment, the appliance 104 orcache manager 232 queues and responds the requests transparently to theclient 102 a-102 n without submitting the request to the server 106a-106 n. The appliance 104 or cache manager 232 may use any mechanismsuch as a queue, object, or data structure in memory to queue therequests from the client 102 a-102 n to be responded to using thetechniques of the present invention.

At step 530, the device 104 determines the object requested by thesecond client 102 b is in the response stored in the buffer or is in theresponse currently being transmitted to the first client 102 a. In someembodiments, the cache manager 232 determines the object is stored inthe buffer. In some cases, the cache manager 232 first checks cachememory or storage for the object, and then checks the buffer for theobject. In other cases, the buffer may be considered a storage locationfor cached objects to be searched or managed by the cache manager 232.In other embodiments, the cache manager 232 requests the packetprocessing engine 240 to check if the object is in the buffer. Inadditional embodiments, the cache manager 232 determines if the objectis in the buffer via any application programming interface (API) of thekernel 204. In some cases, the cache manager 232 interfaces to anydrivers, network handlers, filters, or other software handling orprocessing any portion of the network stack to determine if the objectis in the buffer.

Further at step 530, the present invention provides the object stored inthe buffer or being transmitted as the response to the first client 102a for use in the response to the second request from the second client102B at step 525. In some embodiments, the device 104, such as via thepacket processing engine 104, forms a response to the second request byusing any portion of the buffer and related data structures of thenetwork stack. In one embodiment, the object is stored in the bufferonce, such as in a single network stack configuration, and used to formthe network packets to be transmitted to the first client 102 a andsecond client 102B respectively. In other embodiments, a copy of theobject from the buffer is used to form the response to the secondrequest. In further embodiments, the object is provided from a firstnetwork stack to a second network stack to form the response to thesecond request for the second client 102B. The response of the firstclient 102 a may be maintained in the buffer, and any portion or all ofthe response in the buffer may be used to provide the response for thesecond client 102B. In one embodiment, the response of the first clientstored in the buffer may be used without modification to respond to thesecond client 102B. In another embodiment, the data representing theobject stored in the response of the first client 102 a in the buffer ismodified to provide the response for the second client 102B.

At step 535 of method 500 of the present invention, the first responseto the first client 102 a is transmitted from the device 104, and thesecond response in response to the second request of the second client102B is transmitted from the device 104. The response to the firstclient 102 a and second client 102B may be transmitted in any order fromthe device 104, with any time interval between each other, using thesame network stack 267 or different network stacks. In some embodiments,the response for the second client 102B is also queued, held or storedin a buffer waiting to be transmitted. As such, any additional requestsfor the same object may also be provided via the first response orsecond response stored in the buffer and not yet transmitted, flushed,removed or otherwise unavailable.

At step 540, the response of the first client 102 a is flushed orotherwise removed from the buffer. In some embodiments, if the responseof the second client 102B was stored to the buffer, the response of thesecond client is also removed. In other embodiments, the response of thesecond client may remain in the buffer and steps 525 to 535 of method500 are practiced again using the second client response while receivinga third request for the object from a third client.

In another aspect, the present invention is directed towards a “flashcrowd” technique for handling situations where the appliance 104 orcache manager 232 receives additional requests, e.g., nearlysimultaneous requests, for the same object during the time the server isprocessing and returning the response object for a first requestor. Onceall such nearly simultaneous requests are responded to by the cache, theobject is immediately flushed from the cache memory, with no additionalexpiry time or invalidation action by application or administrator. Thistechnique of the present invention enables data to be cached and servedfor very small amounts of time for objects that would otherwise beconsidered non-cacheable. This approach yields a significant improvementin applications that serve fast changing data to a large volume ofconcurrent users, such, for example, as real time stock quotes, or afast evolving news story.

FIG. 6 describes a method 600 depicting the steps taken by the cache ofthe present invention to serve the same content to each of the clientrequestors as it receives such content in response to the first request.In brief overview of method 600, at step 610, the cache manager 232 orappliance 104 receives a first request from a first client 102 a for anobject, such as a dynamically generated object, from an originatingserver 106 a-106 n. At step 615, the cache forwards the first request tothe originating server 106 a-106 n. While waiting for a response to thefirst request or prior to responding to the first request of the firstclient 102 a, at step 620, the cache receives a second request from asecond client 102 b for the object. In some embodiments, the cache doesnot forward the second request to the originating server but insteadqueues the second request. At step 625, the cache receives the responseto the first request from the originating server, and at step 630,transmits the received response and/or object from the response to thefirst client 102 a in response to the first request, and to the secondclient 102 b in response to the second request. At step 635, the cachemay receive an additional third request from a third client 102 c forthe object prior to completing transmission of the received responseand/or object from the response to either the first client 102 a orsecond client 102 b. At step 640, the cache also transmits the receivedresponse and/or object from the response to the third client 102 c. Uponresponding to the requests, the cache, at step 645, flushes the objectfrom the cache memory, for example, without any expiry time orinvalidation action by the application, originating server, oradministrator.

In further details and in view of FIGS. 1 and 2, step 610 of method 600of the present invention, the cache manager 232 of appliance 104, in oneembodiment, may intercept or otherwise receive a request from a client102 a-102 n, for example client 102 a in FIG. 1 for an object, such as adynamically generated object, at any point in time during communicationsbetween the client 102 a and the originating server, for example 106 a.In one embodiment, the request from the first client 102 a may be thefirst time the first client 102 a has requested the object. In oneexample, the client 102 a may request the object for the first time uponconnection to the origination server 106 a. In another example, thecache manager 232 may have not previously cached or requested the objectfrom the originating server 106 a on behalf of the client 102 a or anyother client 102 a-102 n. In other embodiments, the request from theclient 102 a may be a subsequent request for a previously requestedobject. In yet other embodiments, the cache manager 232 may havepreviously cached the object requested by the client 102 a at step 610.As such, the techniques of the present invention as illustrated in FIG.6 may be practiced at any point during the course of communicationsbetween the clients 102 a-102 n and the originating servers 106 a-106B,regardless if the object is currently cached, has been previously cachedor has never been cached.

At step 615, the cache manager 232 forwards the request from the client102 a to the originating server 106 a. In one embodiment, the cachemanager 232 forwards the request to the originating server withoutchecking if the object is stored in the cache manager 232. In someembodiments, the cache manager 232 first checks if the object requestedby the client 102 a is available from the cache manager 232, and if not,then forwards the request to the originating server 106 a. In otherembodiments, the cache manager 232 checks if the object is stored in thecache but is either invalid or otherwise needs to be updated, or isotherwise about to be invalid or shortly need to be updated. In anotherembodiment, the cache manager 232 determines a dynamically generatedobject stored in the cache should be dynamically regenerated and servedfrom the originating server 106 a-160N. As such, if the cache manager232 determines the object should be requested from an originating server106 a-106 n, the cache manager 232 forwards the request from the client102 a to the originating server 106 a-106 n. In one embodiment, thecache manager 232 forwards the same or original request received fromthe first client 102 a to the originating server 106 a. In anotherembodiment, the cache manager 232 sends a request formed by the cachemanager 232 for the object requested or identified in the request by thefirst client 102 a.

While waiting for a response to the request of the first client 102 afrom the originating server 106 a-160N and/or prior to responding to therequest of the first client 102 a, the cache manager 232, at step 620,may intercept or otherwise receive one or more additional requests forthe object identified, associated with or requested by the first requestof the first client 102 a at step 610. For example, in one embodiment,the cache manager 232 receives a second request from a second client102B for the object, which may be dynamically generated by theoriginating server 106 a in response to the first request. In oneembodiment, the appliance 104 or the cache manager 232 queues the secondrequest, and any additional requests in a queue using any type and formof queuing mechanism. In some embodiments, the second and/or additionalrequests for the object may occur or be received by the cache manager232 nearly simultaneously. In other embodiments, the second and/oradditional requests may be intercepted or received by the cache manager232 at any point between receiving the first request and receiving aresponse to the first request from the originating server 106 a. In yetanother embodiment, the second and/or additional requests may bereceived by the cache manager 232 at any point between receiving thefirst request and providing a response to the first request to theclient 102 a.

In one embodiment, prior to responding to the first request, the cachemanager 232 does not forward the second request from the second client102 b for the object and/or any other additional requests for theobjects from any other clients 102 c-102 n. In one embodiment, the cachemanager 232, at step 625, queues or otherwise holds the second andadditional requests, for example, at the appliance 104, until receivingthe response to the first request. As such, the cache manager 232transparently to the second client 102B does not forward the secondclient's request. However, in some embodiments, it appears to the secondclient 102B that the request is being processed, for example, by anoriginating server 106 a-106 n

In some embodiments, the second request from the second client 102 a mayrequest the same object as the first request at step 610 but alsoinclude an additional request for another object or other information,static, dynamic or otherwise. For example, the request of the secondclient 102B may include multiple requests. In these embodiments, thecache manager 232 may identify the portion of the request related to theobject already requested in connection with the first request of thefirst client 102 a, and not process the identified portion until theresponse to the first request is received. For the other portions of therequest from the second client 102B, the cache manager 232 may processthese other requests in any suitable manner, such as by any techniquesof the present invention described herein. For example, the secondrequest may include a request for a second object, for which the cachemanager 232 requests the second object from an originating server 102a-20B or obtains the second object from the cache manager 232.

At step 625, the method 600 of the present invention receives a responseto the first request forwarded to the originating server 106 a at step615. In some embodiments, the response includes the object, such as anobject dynamically generated by any of the servers 106 a-106 n. In otherembodiments, the response includes a pointer to a location of the objectto be fetched or otherwise obtained by the cache manager 232. In furtherembodiments, the response may indicate some error message or otherwiseindicate the object requested cannot be provided. In one embodiment, thecache manager 232 compares the received response to the correspondingrequest to determine if the response is suitable or appropriate for therequest. In further embodiments, the cache manager 232 obtains contentsof the response, such as the dynamically generated object, and creates,modifies, or otherwise provides the response desired to be communicatedto the first client 102 a.

At step 630, the cache manager 232 of the present invention uses theresponse and/or object from the response received for the first requestto respond to the first request and the second or additional requestsfor the same object. In one embodiment, the cache manager 232 transmitsthe received response to the first client 102 a in response to the firstrequest, and transmits the received response to the second client 102Bin response to the second request. In another embodiment, the cachemanager 232 transmits the received response to the first client 102 a,but modifies the received response and transmits the modified responseto the second client 102B in response to the second request. Forexample, the cache manager 232 may modify the response to the secondclient 102B to include session information or other informationcorresponding to the second request. In further embodiments, the cachemanager 232 may obtain the object requested from the received response,and create, modify or otherwise provide a response corresponding to thefirst request and the second request and including the object, such as afirst response and a second response, and transmit the correspondingresponse to the first client 102 a and second client 102B. As such, thecache manager 232 may use all or any portion of the response receivedfor the first request from an originating server 106 a-106 n inresponding to the requests from the clients 102 a-102 b for the object.

In some embodiments, the cache manager 232 may intercept or otherwisereceive, at step 635, additional requests for the object from otherclients 102 c-102 n after receiving the response to the first requestfrom the server 106 a and before completing transmission of a responseto the first client 102 a, second client 102 b or any other client 102a-102 c requesting the object at step 625. For example, the cachemanager 232 may receive a third request for the object from a thirdclient 102C. In these embodiments, the cache manager 232 may use all orany portion of the response received for the first request from anoriginating server 106 a-106 n and communicated to clients 102 a and/or102 b at step 630 to respond to these additional requests. In anotherembodiment, the cache manager 232 may have completed transmission ofresponses to the first client 102 a and second client 102 b but maystill have any of the responses in memory or otherwise available torespond to the third request of the third client 102 c with the objectprovided via the first request. As such, at step 640, the cache manager232 of the present invention can respond to the third request or furtheradditional requests without requesting the object again from theoriginating server and/or without storing the requested object in cache.

In some embodiments, the appliance 104 or the cache manager 232 queuesthe second request, third request or any additional requests in a queue.As such, in one embodiment, the clients 102 a-102 n do not need toresubmit the requests as the appliance 104 or the cache manager 232queues and responds to the requests on behalf of the servers 106 a-106n. In another embodiment, the appliance 104 or cache manager 232 queuesand responds the requests transparently to the client 102 a-102 nwithout submitting the request to the server 106 a-106 n. The appliance104 or cache manager 232 may use any mechanism such as a queue, object,or data structure in memory to queue the requests from the client 102a-102 n to be responded to using the techniques of the presentinvention. Additionally, the appliance 102 or cache manager 232 mayrespond to the client requests in any order, and not necessarily in theorder received or in a first-in first-out manner from the queue.

At step 645, the cache manager 232 flushes the received response to thefirst request from the originating server 106 a, any responses to theclient's requests, and/or the object requested by the requests from thecache manager 232. In one embodiment, the cache manager 232 immediatelyflushes the response and/or object from cache upon completing thetransmission of the last response, such as at step 630 in one embodimentor step 640, in another embodiment. In some embodiments, the cachemanager 232 maintains the object for a predetermined time period, suchas in accordance with an expiry time. In other embodiments, the cachemanager 232 maintains the object in cache until the originating server106 a-160N invalidates the object. In yet another embodiment, the objectremains in memory or storage until the memory or storage is written overor otherwise used by the cache manager 232 for other purposes.

Although the techniques of methods 500 and 600 of the present inventionare generally described above in regards to a request for a singleobject from multiple clients, those ordinarily skilled in the art willrecognize and appreciate that the flash cache and flash crowd handlingtechniques of the present invention may be used to handle multipleobject requests either in a single request or a series of requests frommultiple clients that may occur either sequentially, concurrently, ornearly concurrently or simultaneously. As such, multiple instances ofpracticing methods 500 and 600 may be executing at various frequenciesand at various times during the course of operation of the cache manager232 or appliance 104 in a network environment 200. Furthermore, eachrequest or series of requests from each client may request differentsets of objects, with a common set of one or more objects requestedamong one or more clients.

For example in view of the flash crowd techniques of the presentinvention, a first client may request a first object and a secondobject, while a second client may request the second object and a thirdobject, and the third client may request the first object and the thirdobject. In some embodiments, the cache of the present invention usesmultiplexing and/or demultiplexing techniques to request an object oncefor each object of multiple requests outstanding between multipleclients, and provide the corresponding response to each clientaccordingly. Further to the above example, the cache may obtain thefirst object from the originating server once for the first client andthe third client, the second object once for the first client and secondclient, and the third object once for the second client and thirdclient. In some embodiments, when all the objects for one or morerequests for a client are received by the cache, the cache transmits thecorresponding response or responses to the client. In other embodiments,the cache may respond to the client with each object upon receipt of theobject by the cache.

Likewise, in view of the flash cache technique of method 500 of thepresent invention, the buffer of the appliance may comprise multipleobjects waiting to be transmitted while additional requests for thoseobjects are received by the appliance. In one embodiment, the applianceof the present invention uses a single TCP/IP stack from which multiplebuffers are associated with and temporarily store multiple objectsduring the course of transmission. As such, a first object in the buffermay be used to respond to requests from a first client and a secondclient for the first object, and a second object in the buffer may beused to respond to requests from a third client and fourth client forthe second object, or to respond to a request from either or both of thefirst client or second client for the second object.

Furthermore, the flash cache and flash crowd techniques of the presentinvention may be practiced in conjunction or in any combination witheach other as those ordinarily skilled in the art will recognize andappreciate. For example, the appliance of the present invention may beproviding responses to additional requests for an object from the buffervia the flash cache technique, but determines the object in the bufferis either invalid or otherwise needs to be requested from theoriginating server. At that point, the appliance may switch to the flashcrowd technique by requesting the object from the originating server.The appliance then uses the object requested from the originating serverto respond to additional requests for the object received and queued atthe appliance while waiting for the originating server to respond.

12. Etag Insertion

In another aspect, the present invention is directed towards usingtechniques to increase cache hit rates by inserting information, such asentity tag and cache control information for an object, in a response toa client to enable the cache to check for a hit in a subsequent request.The present invention provides a technique for identifying a dynamicallygenerated object as cacheable to a client when the originating serverdid not identify the object as cacheable. In some embodiments, such asan embodiment handling HTTP requests and responses for objects, thetechniques of the present invention insert an entity tag, or “etag” intothe response to provide cache control for objects provided withoutentity tags and/or cache control information from an originating server.In brief reference to the HTTP protocol as known to those ordinarilyskilled in the art, an “etag” is the ETag header field of an HTTP basedmessage which provides the current value of the entity tag for arequested variant, such as an object, and may be used for comparisonwith other entities from the same resource. The etag provides amechanism for validation in the HTTP protocol. Also, an HTTP basedmessage may include a Cache-Control header field to specify directivesto caching mechanisms along the request/response message orcommunication chain. The cache-control directives specify desiredcaching behavior.

Referring now to FIGS. 7A and 7B and in view of FIGS. 1 and 2, steps aredepicted for an embodiment of practicing the “etag” techniques of thepresent invention. In brief overview of FIG. 7A, method 700 of thepresent invention, step 710, receives a response having an objectrequested by a client 102 a-102 n, such as object served from anoriginating server 106 a-106 n and received by a cache manager 232 ofthe appliance 104. At step 715, the cache manager 232 determines if theobject from the response is under tag entity control or otherwise has anentity tag. If the object does not have an entity tag, then at step 720,the cache manager 232 generates an entity tag for the object. At step725, the cache manager 232 modifies the response to provide entitycontrol information including the cache generated entity tag. At step730, the cache manager 232 stores the object with the entity taginformation to the cache manager 232. At step 735, the cache manager 232responds to the client 102 a-102 n requesting the object with themodified response. As such, the client 102 a-102 n receives an entitytag controlled object from the cache manager 232 in a manner transparentto the client 102 a-102 n, such that the client 102 a-102 n is not awarethat the originating server 106 a-106 n did not provide the object withthe entity tag information.

In further detail, at step 710 of the present invention, the cachemanager 232 receives a response to a request of a client 102 a-102 n forthe object from an originating server 106 a-106 n. In some embodiments,the response may be received from a request sent from the client 102a-102 n to the server 106 a-106 n, or from the cache manager 232 onbehalf of the client 102 a-102 n to the server 106 a-106 n. In oneembodiment, the cache manager 232 forwards the request from the client102 a-102 n to the server 106 a-106 n after checking the cache manager232 for a matching and/or valid object. In another embodiment, the cachemanager 232 requests the object from the server 106 a-106 n duringpracticing the flashing crowd technique of the present inventiondescribed in conjunction with FIG. 6.

At step 715 of method 700, the cache manager 232 determines if theobject received in the response at step 710 is under entity tag controlor has an entity tag. In some embodiments, the cache manager 232inspects, examines, checks or otherwise reads any field, header, data,or other information of the one or more packets providing the response.As those ordinarily skilled in the art will recognize and appreciate,the portion of the packet or packets to inspect or read depends on thetype of one or more protocols used for the response. In one embodiment,the cache manager 232 determines the entity tag portion of the responsepacket is missing, empty or otherwise not provided. In anotherembodiment, the cache manager 232 determines the entity tag portion ofthe response packet is corrupted, invalid or otherwise not useable bythe cache manager 232 and/or the client 102 a-102B. In some embodiments,the cache manager 232 determines the response has an entity tag or isunder entity control but not in the type, form or manner desired by thecache manager 232. In addition to or instead of the entity tag, thecache manager 232, in a similar manner to the entity tag, determines ifthere exists desired cache-control information in the response. Infurther embodiments, the cache manager 232 may determine via the policyengine 236 if the object should be under entity tag and/or cachecontrol. As such, in some embodiments, the cache manager 232 may notcontinue performing the etag techniques of the present invention, suchas steps 720 to 725, if the appliance 104 determines via the policyengine 236 the object should not be under entity tag or cache control.

At step 720, the cache manager 232 generates desired entity tag andcache control information for the object of the received response. Theetag may comprise any type and/or form of entity tag representation andmay comprise any numeric, alphanumeric, letters in one or morecharacters. In the embodiment of HTTP, the etag may comprise a string orquoted string, and may further include a prefix for indicating the etagis weak or strong in accordance with the HTTP protocol. In someembodiments, the cache manager 232 uses an entity header, or etag,counter which increments an etag counter sequentially, or in any desiredincrements. In some embodiments, the etag counter is global to allobjects for which the cache manager 232 generates an etag. In otherembodiments, the cache manager 232 may have multiple etag counters, eachassociated with or specific to an object or groups of objects.

In one embodiment, the etag may comprise a universal identifier (UID) orglobal universal identifier (GUID) associated with the object. Inanother embodiment, the etag may comprise an incarnation number aspreviously described above. In some embodiments, the cache manager 232obtains the etag from an application, program, or other form ofexecutable instructions running on the appliance 104 or in otherembodiments, from another system or device accessible by the appliance104 via the network. In further embodiments, the cache manager 232 mayuse any suitable algorithm, business rule or logic to generate a desiredunique etag for the object. In one embodiment, the appliance 104comprises a database, file, or other organized storage element forgenerating and maintaining etags for objects for the cache manager 232.

At step 725 of method 700, the cache manager 232 modifies the responsefrom the originating server 106 a-106 n to include desired entity tagand/or cache control information, such as the entity tag generated bythe cache at step 720. In one embodiment, the cache manager 232 insertsthe entity tag information via an entity tag header of the protocol,such as the HTTP protocol. In another embodiment, the cache manager 232modifies any entity tag header fields in the response to include thedesired entity tag. In yet another embodiment, the cache manager 232replaces the entity tag fields in the response with the entity tagfields desired to be used by the cache manager 232. In furtherembodiments, the entity tag information is inserted, modified or placedinto any portion of the network packet or packets for communicating theresponse to the client 102 a-102 n.

Additionally, step 725, the cache manager 232 may insert, modify orotherwise place any desired cache-control information into any header,fields of a protocol, or any other portion of network packets forcommunicating the response. The cache-control information may begenerated by the cache manager 232 by any suitable means and/ormechanisms. In one embodiment, the cache-control information isconfigured into and obtained via the policy engine 236 of the device104. In another embodiment, the cache manager 232 determines the desiredcache-control information from any algorithms, business rules, or logicof the cache manager 232. In some embodiments, the cache-controlinformation may be based on any history of the network and cachingperformance between the client 102 a-102 n and the appliance 104 orcache manager 232.

At step 730 of the method 700 of the present invention, the cachemanager 232 stores the generated entity tag with the object in anysuitable or desired manner in the cache manager 232. In someembodiments, the entity tag is included in or is made a part of thestored object. In other embodiments, the entity tag is stored separatelybut associated with the object. In another embodiment, the entity tagand object are associated via a database, linked list, lookup table, orany other mechanism for associating one entity with another entity asknown to those skilled in the art. Additionally, the cache-controlinformation provided with the object and/or entity tag, in someembodiments, may also be stored in association with the object in thecache manager 232.

At step 735, the cache manager 232 causes the appliance 104 to transmitthe modified response to the client 106 a-106 n. For example, the cachemanager 232 requests the packet processing engine 240 to transmit themodified response to the client 102 a-102 n. As such, the client 102a-102 n receives the object with entity tag and cache controlinformation, even though unbeknownst to the client 102 a-102 n theoriginating server 106 a-106 n did not generate such information in oneembodiment, or the cache manager 232 changed such information generatedby the server 106 a-106 n. The etag technique of the present inventioncan ensure that every object has a validator, i.e., an etag, or has anetag controlled or otherwise desired by the cache manager 232. If anobject doesn't have a validator or a desired validator, then the cachemanager 232 inserts its own etag. These etag validators can avoidserving full responses on repeat requests for an object as will bediscussed in connection with FIG. 7B.

FIG. 7B depicts a method 750 of the present invention for using theentity tag to check by a client 102 a-102 n if an object has beenmodified. As such, the client 102 a-102 n can cache the object on theclient 102 a-102 n and check with the cache manager 232 or originatingserver 106 a-106 n if the object has changed. In brief overview ofmethod 750, at step 775, the cache manager 232 receives a request fromthe client 102 a-102 n to check the entity tag of an object. At step760, the cache compares the entity tag received from the request of theclient 102 a-102 n with the entity tag of object stored in cache. Atstep 765, the cache manager 232 determines if the entity tag from theclient matched the entity of the object in cache. If the entity tags donot match, then at step 770, the cache manager 232 responds to theclient 102 a-102 n with a response providing the current entity tag forthe object. If the entity tags do match, then at step 775, the cachemanager 232 responds to the client 102 a-102 n with a responseindicating the object has not been modified.

In further detail, at step 755, the cache manager 232 may receive arequest from the client 102 a-102 n to check or validate the entity tagof an object by any type and/or form of communication. In oneembodiment, the request comprises an HTTP message with an If-Match orIf-None-Match request header field, which includes an entity tag stringvalue provided by the client 102 a-102 n. For example, the client 102a-102 n may have a copy of the object in a client cache or otherwiseavailable on the client 102 a-102 n. The client 102 a-102 n uses theentity tag of the client's copy of the object in the request todetermine or validate if the object is current.

At step 760, the cache manager 232 compares the entity tag valuereceived from the client 102 a-102B with the entity tag for the objectto determine if the cache manager 232 has a different version of theobject than the client 102 a-102 n. The cache manager 232 may use anysuitable means and/or mechanisms to compare the entity tag value fromthe client 102 a-102 n with the entity tag value for the cached object.In one embodiment, the cache manager 232 performs a byte-compare on theetag value from the request with the etag stored with the object in thecache. In another embodiment, the cache manager 232 performs and useshash algorithms on the entity tag and/or the cache object to determineif the object has been modified. In further embodiments, the cachemanager 232 requests the object from the server 106 a-106 n, andcompared the received object with the cached object to determine if theobject has been modified.

If, at step 765, the entity tag from the client 102 a-102 n does notmatch the entity tag stored with the object in the cache manager 232then at step 770, the cache manager 232 sends a response to the clientindicating the entity tags do not match or otherwise the object has beenmodified. In one embodiment, the cache manager 232 sends a response tothe client 102 a-102 n providing a new entity tag value for the object.In another embodiment, the cache manager 232 sends a response to theclient 102 a-102 n provide a new version of the object, or the modifiedobject, such as a modified object with a new entity tag stored in thecache manager 232. If, at step 765, the entity tag from the client 102a-102 n does match the entity tag stored with the object in the cachemanager 232 then at step 775, the cache manager 232 sends a response tothe client indicating the entity tags do match or otherwise the objecthas not been modified.

Although the etag techniques of the present invention has generally beendescribed in reference to the HTTP protocol, those ordinarily skilled inthe art will recognize and appreciate the present invention may bepracticed with any type and/or form of protocol to provide entity tagand cache information. In some embodiments, the protocol may not supportentity tag and cache information fields, while in other embodiments, theprotocol may support entity tag and cache information fields. In any ofthese embodiments, the techniques of the present invention provideentity tag and cache control information when the server does notprovide this information in a response or when the cache decides tochange or control the entity and cache control information provided bythe server in a manner controlled or desired by the appliance 104.

In order to address the challenge to dynamically caching contentdynamically generated by origin servers and applications the cacheappliance of the present invention, can incorporate pre-packagedpolicies for certain commonly used applications such as Outlook® WebAccess, Oracle® Configurator, and Serena® TeamTrack®. These packages aredesigned with an understanding of the way such applications process datathereby enabling the cache to make more intelligent decisions aboutwhich object and how long to keep objects generated by such applicationsstored in the cache memory.

D. Exemplary Computer System-Based Implementation

The functions of the present invention may be implemented usinghardware, software, or a combination thereof and may be implemented inone or more computing devices or other processing systems. For example,FIGS. 8A AND 8B depict an example computing device 800 that may beutilized to implement any of the techniques, methods and functions ofthe present invention. As such, the appliance 104 of the presentinvention may be a computing device 800 or a specific purpose devicedesigned and constructed to provide any of the techniques, functions andoperations of the present invention described herein.

FIGS. 8A and 8B depict block diagrams of a computing device 800, and insome embodiments, also referred to as a network device 800, useful forpracticing an embodiment of the present invention. As shown in FIGS. 8Aand 8B, each computing device 800 includes a central processing unit802, and a main memory unit 822. As shown in FIG. 8A, a typicalcomputing device 800 may include a visual display device 824, a keyboard826 and/or a pointing device 827, such as a mouse. Each computing device800 may also include additional optional elements, such as one or moreinput/output devices 830 a-830 b (generally referred to using referencenumeral 830), and a cache memory 840 in communication with the centralprocessing unit 802.

The central processing unit 802 is any logic circuitry that responds toand processes instructions fetched from the main memory unit 822. Inmany embodiments, the central processing unit is provided by amicroprocessor unit, such as: those manufactured by Intel Corporation ofMountain View, Calif.; those manufactured by Motorola Corporation ofSchaumburg, Ill.; those manufactured by Transmeta Corporation of SantaClara, Calif.; those manufactured by International Business Machines ofWhite Plains, N.Y.; or those manufactured by Advanced Micro Devices ofSunnyvale, Calif. The computing device 800 may be based on any of theseprocessors, or any other processor capable of operating as describedherein.

Main memory unit 822 may be one or more memory chips capable of storingdata and allowing any storage location to be directly accessed by themicroprocessor 802, such as Static random access memory (SRAM), BurstSRAM or SynchBurst SRAM (BSRAM), Dynamic random access memory (DRAM),Fast Page Mode DRAM (FPM DRAM), Enhanced DRAM (EDRAM), Extended DataOutput RAM (EDO RAM), Extended Data Output DRAM (EDO DRAM), BurstExtended Data Output DRAM (BEDO DRAM), Enhanced DRAM (EDRAM),synchronous DRAM (SDRAM), JEDEC SRAM, PC100 SDRAM, Double Data RateSDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), SyncLink DRAM (SLDRAM),Direct Rambus DRAM (DRDRAM), or Ferroelectric RAM (FRAM). The mainmemory 822 may be based on any of the above described memory chips, orany other available memory chips capable of operating as describedherein. In the embodiment shown in FIG. 8A, the processor 802communicates with main memory 822 via a system bus 850 (described inmore detail below). FIG. 8A depicts an embodiment of a computing device800 in which the processor communicates directly with main memory 822via a memory port 803. For example, in FIG. 8B the main memory 822 maybe DRDRAM.

FIG. 8B depicts an embodiment in which the main processor 802communicates directly with cache memory 840 via a secondary bus,sometimes referred to as a backside bus. In other embodiments, the mainprocessor 802 communicates with cache memory 840 using the system bus850. Cache memory 840 typically has a faster response time than mainmemory 822 and is typically provided by SRAM, BSRAM, or EDRAM.

In the embodiment shown in FIG. 8A, the processor 802 communicates withvarious I/O devices 830 via a local system bus 850. Various busses maybe used to connect the central processing unit 802 to any of the I/Odevices 830, including a VESA VL bus, an ISA bus, an EISA bus, aMicroChannel Architecture (MCA) bus, a PCI bus, a PCI-X bus, aPCI-Express bus, or a NuBus. For embodiments in which the I/O device isa video display 824, the processor 802 may use an Advanced Graphics Port(AGP) to communicate with the display 824. FIG. 8B depicts an embodimentof a computer 800 in which the main processor 802 communicates directlywith I/O device 830 b via HyperTransport, Rapid I/O, or InfiniBand. FIG.8B also depicts an embodiment in which local busses and directcommunication are mixed: the processor 802 communicates with I/O device830 a using a local interconnected bus while communicating with I/Odevice 830 b directly.

The computing device 800 may support any suitable installation device816, such as a floppy disk drive for receiving floppy disks such as3.5-inch, 5.25-inch disks or ZIP disks, a CD-ROM drive, a CD-R/RW drive,a DVD-ROM drive, tape drives of various formats, USB device, hard-driveor any other device suitable for installing software and programs suchas any software 820, or portion thereof, related to the presentinvention or otherwise providing any of the techniques of the presentinvention. The computing device 800 may further comprise a storagedevice 828, such as one or more hard disk drives or redundant arrays ofindependent disks, for storing an operating system and other relatedsoftware, and for storing application software programs such as anyprogram related to the software 820 of the present invention.Optionally, any of the installation devices 816 could also be used asthe storage device 828.

Furthermore, the computing device 800 may include a network interface818 to interface to a Local Area Network (LAN), Wide Area Network (WAN)or the Internet through a variety of connections including, but notlimited to, standard telephone lines, LAN or WAN links (e.g., 802.11,T1, T3, 56 kb, X.25), broadband connections (e.g., ISDN, Frame Relay,ATM), wireless connections, or some combination of any or all of theabove. The network interface 818 may comprise a built-in networkadapter, network interface card, PCMCIA network card, card bus networkadapter, wireless network adapter, USB network adapter, modem or anyother device suitable for interfacing the computing device 800 to anytype of network capable of communication and performing the operationsdescribed herein.

A wide variety of I/O devices 830 a-830 n may be present in thecomputing device 800. Input devices include keyboards, mice, trackpads,trackballs, microphones, and drawing tablets. Output devices includevideo displays, speakers, inkjet printers, laser printers, anddye-sublimation printers. The I/O devices may be controlled by an I/Ocontroller 823 as shown in FIG. 8A. The I/O controller may control oneor more I/O devices such as a keyboard 826 and a pointing device 827,e.g., a mouse or optical pen. Furthermore, an I/O device may alsoprovide storage 828 and/or an installation medium 816 for the computingdevice 800. In still other embodiments, the computing device 800 mayprovide USB connections to receive handheld USB storage devices such asthe USB Flash Drive line of devices manufactured by Twintech Industry,Inc. of Los Alamitos, Calif.

In further embodiments, an I/O device 830 may be a bridge 870 betweenthe system bus 850 and an external communication bus, such as a USB bus,an Apple Desktop Bus, an RS-232 serial connection, a SCSI bus, aFireWire bus, a FireWire 800 bus, an Ethernet bus, an AppleTalk bus, aGigabit Ethernet bus, an Asynchronous Transfer Mode bus, a HIPPI bus, aSuper HIPPI bus, a SerialPlus bus, a SCI/LAMP bus, a FibreChannel bus,or a Serial Attached small computer system interface bus.

A computing device 800 of the sort depicted in FIGS. 8A and 8B typicallyoperate under the control of operating systems, which control schedulingof tasks and access to system resources. The computing device 800 can berunning any operating system such as any of the versions of theMicrosoft® Windows operating systems, the different releases of the Unixand Linux operating systems, any version of the Mac OS® for Macintoshcomputers, any embedded operating system, any network operating system,any real-time operating system, any open source operating system, anyproprietary operating system, any operating systems for mobile computingdevices or network devices, or any other operating system capable ofrunning on the computing device and performing the operations describedherein. Typical operating systems include: WINDOWS 3.x, WINDOWS 95,WINDOWS 98, WINDOWS 2000, WINDOWS NT 3.51, WINDOWS NT 4.0, WINDOWS CE,and WINDOWS XP, all of which are manufactured by Microsoft Corporationof Redmond, Wash.; MacOS, manufactured by Apple Computer of Cupertino,Calif.; OS/2, manufactured by International Business Machines of Armonk,N.Y.; and Linux, a freely-available operating system distributed byCaldera Corp. of Salt Lake City, Utah, or any type and/or form of a Unixoperating system, among others.

In other embodiments, the computing device 800 may have differentprocessors, operating systems, and input devices consistent with thedevice. The computing device 800 can be any workstation, desktopcomputer, laptop or notebook computer, server, handheld computer, mobiletelephone, any other computer, or other form of computing ortelecommunications device that is capable of communication and that hassufficient processor power and memory capacity to perform the operationsof the present invention described herein. Moreover, the computingdevice 800 can be any type and/form of network device, such as a remoteaccess device, a Virtual Private Network (VPN) device, a Secure SocketLayer (SSL) VPN device, router, switch, bridge, or other network devicein any form capable of performing the operations of the presentinvention described herein

Although the appliance of the present invention is generally discussingusing or communicating via TCP/IP, the appliance may use or communicatewith any modified transport control protocol, such as Transaction TCP(T/TCP), TCP with selection acknowledgements (TCP-SACK), TCP with largewindows (TCP-LW), congestion prediction such as in the TCP-Vegasprotocol, and TCP spoofing. Additionally, the appliance of the presentinvention may use any internode or high-performance protocol. Moreover,the appliance of the present invention may use or operate with any othertransport and network protocols, such as Sequenced packet exchange (SPX)protocol over the Internetwork Packet Exchange (IPX) protocol.

CONCLUSION

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample only, and not limitation. Thus, it will be understood by thoseskilled in the relevant art(s) that various changes in form and detailsmay be made therein without departing from the spirit and scope of theinvention as defined in the appended claims. Accordingly, the breadthand scope of the present invention should not be limited by any of theabove-described exemplary embodiments, but should be defined only inaccordance with the following claims and their equivalents.

1-30. (canceled)
 31. A method for handling an additional request for anobject while transmitting a response to a requestor of the object,comprising: receiving, by a device from a server, a dynamicallygenerated object served by the server responsive to a first request of afirst client for the dynamically generated object; generating, by thedevice, a first one or more network packets for a first response to thefirst request of the first client, the first response comprising thedynamically generated object; placing, by the device, the first one ormore network packets into a transmission buffer for transmission to thefirst client; receiving, by the device, a second request from a secondclient for the dynamically generated object prior to completingtransmission of the first one or more network packets to the firstclient; and generating, by the device, from the first one or morenetwork packets in the transmission buffer, a second one or more networkpackets for a second response to the second request of the secondclient, the second response comprising the dynamically generated object.32. The method of claim 31, wherein the dynamically generated object isnot cacheable.
 33. The method of claim 32, wherein the dynamicallygenerated object is not stored in a cache of the device.
 34. The methodof claim 31, further comprising holding the first one or more networkpackets in the transmission buffer responsive to networkcharacteristics.
 35. The method of claim 31, further comprisingbeginning transmission of the first one or more network packets to thefirst client prior to receiving the second request from the secondclient.
 36. The method of claim 31, further comprising determining, bythe device, responsive to receiving the second request, that the firstone or more network packets comprising the dynamically generated objectare in the transmission buffer.
 37. The method of claim 31, furthercomprising determining, by the device, the second request is for thedynamically generated object of the first request via an objectdeterminant associated with the second request.
 38. The method of claim31, further comprising removing the first one or more network packetsfrom the transmission buffer, responsive to completing transmission ofthe first one or more network packets to the first client.
 39. Themethod of claim 31, further comprising transmitting the second one ormore network packets to the second client.
 40. The method of claim 31,wherein the transmission buffer comprises one of a TCP/IP buffer or aportion of a TCP/IP stack of the device.
 41. A system for handling anadditional request for an object while transmitting a response to arequestor of the object, comprising: a device in communication with afirst client, a second client, and a server, comprising a transmissionbuffer, the device configured for: receiving, from the server, adynamically generated object served by the server responsive to a firstrequest of the first client for the dynamically generated object,generating a first one or more network packets for a first response tothe first request of the first client, the first response comprising thedynamically generated object, placing the first one or more networkpackets into the transmission buffer for transmission to the firstclient, receiving a second request from the second client for thedynamically generated object prior to completing transmission of thefirst one or more network packets to the first client, and generating,from the first one or more network packets in the transmission buffer, asecond one or more network packets for a second response to the secondrequest of the second client, the second response comprising thedynamically generated object.
 42. The system of claim 41, wherein thedynamically generated object is not cacheable.
 43. The system of claim42, wherein the device further comprises a cache, and the dynamicallygenerated object is not stored in the cache of the device.
 44. Thesystem of claim 41, wherein the device is further configured for holdingthe first one or more network packets in the transmission bufferresponsive to network characteristics.
 45. The system of claim 41,wherein the device is further configured for beginning transmission ofthe first one or more network packets to the first client prior toreceiving the second request from the second client.
 46. The system ofclaim 41, wherein the device is further configured for determining,responsive to receiving the second request, that the first one or morenetwork packets comprising the dynamically generated object are in thetransmission buffer.
 47. The system of claim 41, wherein the device isfurther configured for determining the second request is for thedynamically generated object of the first request via an objectdeterminant associated with the second request.
 48. The system of claim41, wherein the device is further configured for removing the first oneor more network packets from the transmission buffer, responsive tocompleting transmission of the first one or more network packets to thefirst client.
 49. The system of claim 41, wherein the device is furtherconfigured for transmitting the second one or more network packets tothe second client.
 50. The system of claim 41, wherein the transmissionbuffer comprises one of a TCP/IP buffer or a portion of a TCP/IP stackof the device.